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14.  Abstract 


>This  report  describes  a  small  sample  study  of  the  possible  impact 
of  altering  the  separation  minimum  between  aircraft  approaches  to 
dependent  parallel  runways.  The  current  standard  is  2  nautical 
miles  (nmi)  and  the  proposed  new  standard  is  1.5  nmi.  Four  full 
performance  level  air  traffic  controllers  participated  in  12 
hours  of  simulated  air  traffic  control  activity  in  which 
separation  standards  were  altered  in  a  balanced  fashioned  after 
each  1  hour  block  of  simulation.  Data  were  collected  on  multiple 
airspace  and  operator  performance  variables.  Also  collected  were 
workload  and  observer  estimates.  The  goal  was  to  determine  if 
system  performance  could  be  improved  without  compromising  safety. 
Results  indicated  an  increased  frequency  of  landings  using  the 
1.5  nmi  standard  indicating  a  finite  increase  in  airport 
capacity.  There  were  no  indications  of  reduced  safety  or 
increased  operator  workload.  Since  the  data  were  generated  based 
on  a  small  sample,  results  should  be  considered  indicative  rather 
than  conclusive.  / 
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EXECUTIVE  SUMMARY 


Our  limited  airspace  is  becoming  crowded  as  demands  from  the 
flying  public  increase.  This  is  particularly  true  in  the 
terminal  control  areas  of  our  major  airports.  The  construction 
of  new  airports  has  been  minimal  over  the  past  decade,  and  yet 
the  number  of  operations  keeps  mounting.  The  Federal  Aviation 
Administration  (FAA)  is  responsible  for  maintaining  safe  and 
expeditious  travel.  One  of  the  options  available  in  order  to 
move  more  traffic  is  to  reexamine  procedures.  We  need  to 
determine  if  adjustments  or  modifications  can  positively 
influence  productivity  without  overloading  the  system. 

This  study  was  an  effort  to  use  real-time,  person  in  the  loop, 
air  traffic  control  (ATC)  simulation  to  evaluate  the  potential 
impact  of  altering  the  minimum  separation  between  approaches  to 
dependent  parallel  runways  at  a  major,  high  traffic  volume 
airport.  The  current  minimum  is  2  nautical  miles  (nmi) ,  and  it 
has  been  proposed  to  reduce  this  distance  to  1.5  nmi.  The  use  of 
simulation  provided  an  opportunity  to  study  system  operation  and 
operator  performance  in  a  safe  and  controlled  environment. 

This  was  a  small  sample,  preliminary  study  which  involved  the 
participation  of  four  current  terminal  controllers  along  with  two 
control  supervisors,  who  served  as  observer/ evaluators.  All 
participants  were  volunteers  who  came  to  the  FAA  Technical  Center 
for  1  week  of  simulation.  They  were  given  a  period  of  training 
and  familiarization.  The  four  participant  controllers  were 
organized  into  six  operational  teams,  and  team  membership  was 
rotated  after  each  hour  of  simulation.  All  simulation  was 
accomplished  in  1-hour  blocks  with  each  block  being  a  new  problem 
with  its  own  traffic  sample  based  on  the  typical  traffic  from  the 
participants'  home  facility.  The  research  design  called  for  the 
teams  to  work  the  final  approach  position  of  their  airport  under 
two  separation  minima:  2.0  and  1.5  nmi.  The  approach  minimum 
was  alternated  every  hour  of  simulation.  Every  controller  team 
experienced  each  separation  condition  twice.  At  any  one  time, 
two  teams  were  working  independently  on  their  own  simulations. 
During  the  12  hours  of  running  time,  the  equivalent  of  24  hours 
worth  of  simulation  data  were  collected. 

Data  included  both  objective  measures  of  system  operations  and  a 
series  of  subjective  measures.  The  latter  data  included  post- 
irun  questionnaires  and  observer  ratings  and  commentary.  The 
results  of  this  study  are  summarized  in  the  subsequent 
paragraphs . 

The  first  data  to  be  analyzed  was  the  frequency  of  violations  of 
the  airspace  minima.  Longitudinal  violations  involved  aircraft 
that  are  in  trail  of  each  other.  There  were  significantly  more 
violations  for  the  1.5  nmi  standard  during  the  first  attempts  by 
controllers  in  the  simulation.  However,  this  washed  out  by  the 
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second  replication  at  which  time  there  was  absolutely  no 
difference  in  violation  frequencies  for  the  two  standards.  The 
results  for  parallel  violations  were  very  clear.  There  was  a 
very  strong  effect  separation  standard.  The  1.5  nmi  separation 
had  significantly  fewer  violations.  Based  on  a  reanalysis  of  the 
1.5  nmi  data  runs  using  the  2  nmi  filter,  it  was  apparent  that 
controllers  were  not  erring  on  the  side  of  being  overly  cautious 
during  the  1,5  nmi  runs.  They  were  doing  their  best  to  get  the 
most  of  the  system  given  the  rules  in  force. 

A  critical  measure  of  terminal  capacity  is  the  number  of  aircraft 
that  controllers  can  land  in  a  finite  period  of  time.  There  was 
a  significant  difference  between  the  two  separation  minima. 

Using  the  1.5  nmi  standard,  controllers  landed  an  average  of  68 
aircraft  per  hour.  This  was  significantly  more  than  when  using 

the  2  nmi  standard  with  which  they  landed  64.33  per  hour. _ This 

represents  an  increase  of  approximately  5.7  percent.  This  was 
accomplished  with  no  apparent  decrease  in  safety  as  judged  by  an 
analysis  of  the  aircraft  proximity  index  (API) . 

The  two  observers  watched  and  recorded  their  ratings  very 
carefully.  They  rated  workload  and  performance  on  a  case  by  case 
basis  and  identified  no  systematic  differences  based  on 
separation  standards.  Both  observers  commented  on  the  importance 
of  the  working  relationship  between  members  of  the  control  team 
and  the  role  of  strategy  in  establishing  a  viable  parallel 
approach  flow  or  stagger. 

The  post-run  questionnaire  was  administered  to  the  participants 
at  the  end  or  each  hour  of  simulation.  Results  indicated  that 
controllers  felt  that  their  workload  was  moderate  regardless  of 
the  separation  standard,  with  the  exception  of  one  run  in  which  a 
sequence  of  events  not  related  to  the  separation  standard  drove 
the  workload  up  and  the  performance  down.  They  rated  their  own 
performance  as  consistently  good,  which  was  in  contrast  to  the 
observers  who  tended  to  exhibit  more  variability  across  runs. 
Controllers  indicated  that  they  were  busy  but  not  too  busy,  and 
that  their  stress  was  moderate  but  capable  of  running  higher,  as 
in  the  run  when  things  began  to  deteriorate.  Controllers 
indicated  that  both  separation  standards  were  workable  in  their 
facility.  As  with  the  observers,  the  controllers  saw  no 
systematic  differences  based  on  separation  standard  alone. 

During  the  post-experiment  interview,  all  the  controllers  and  the 
observers  expressed  a  willingness  to  comfortably  use  the  1.5  nmi 
separation  standard.  They  agreed  that  it  would  not  compromise 
safety.  Although  it  was  not  clear  to  them  whether  it  would 
reduce  their  workload,  no  one  indicated  that  it  would  cause  them 
any  sort  of  a  workload  problem.  They  rated  the  simulation  as 
moderately  realistic.  Again  during  the  interview,  both 
participants  and  observers  cited  the  importance  of  controllers 
working  together  to  establish  operational  strategy  for  the 
effective  control  of  aircraft  in  the  airspace. 
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INTRODUCTION 


PURPOSE. 

The  purpose  of  this  study  was  to  explore,  through  real-time 
simulation,  the  effects  of  reducing  the  spacing  between  aircraft 
flying  dependent  parallel  approaches  to  adjacent  runways  from  the 
current  minimum  of  2.0  to  1.5  nautical  miles  (nmi) .  The  possible 
impact  of  this  alteration  included  changes  in  the  nature  and 
quality  of  system  operations  and  in  the  workload  and  performance 
of  the  human  operators-the  air  traffic  control  (ATC)  specialists. 

BACKGROUND. 

Currently,  dependent  parallel  operations  are  permitted  on  runways 
that  are  separated  by  2500  feet  or  more.  Aircraft  on  adjacent 
approaches  must  be  separated  diagonally  by  a  minimum  of  2  nmi 
separation.  They  must  also  be  separated  longitudinally  using  a 
series  of  criteria  based  on  the  type  of  aircraft  that  are  in 
trail  of  each  other.  This  separation  is  accomplished  by 
controllers.  The  ability  to  do  this  is  developed  by  experience. 

There  is  a  growing  interest  in  reducing  the  separation  between 
aircraft  approaching  adjacent  runways  using  dependent  parallel 
approaches  operating  under  instrument  meteorological  conditions 
(IMC) .  It  is  possible  that  if  the  current  2  nmi  requirement 
could  be  reduced  to  1.5  nmi  with  no  degradation  of  safety,  then 
capacity  and/or  controller  efficiency  might  be  increased. 

Accordingly,  a  proposal  was  been  made  to  test/ evaluate  the 
potential  generated  by  providing  a  minimum  of  1.5  nmi  radar 
separation  between  aircraft  on  adjacent  parallel  instrument 
landing  system  (ILS)  final  approach  courses.  The  principal 
questions  to  be  answered  were  whether  or  not  decreased  separation 
standards  lead  to  positive  effects  for  airport  operations,  and  to 
evaluate  the  extent  these  effects  had  on  controller  behavior  and 
subjective  perceptions  of  workload. 

OBJECTIVE* 

The  specific  objectives  of  this  investigation  were  to: 

1.  Determine  if  more  aircraft  can  be  landed  when  controllers 
maintain  1.5  nmi  separation  than  when  they  are  trying  to  maintain 
2.0  nmi. 

2.  Determine  whether  present  levels  of  safety  are  retained  when 
the  reduced  separation  is  used. 

3.  Determine  if  the  reduction  of  separation  has  any  impact  on 
the  controllers*  ability  to  maintain  separation  for  aircraft  in 
trail  on  the  same  approach. 
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4.  Determine  if  a  change  in  separation  standard  affects 
controller  work  effort  and  if  so,  how. 

SIMULATION  ENVIRONMENT. 

This  effort  is  intended  to  simulate  a  realistic  ATC  environment 
using  dependent  parallel  approaches.  For  this  reason,  Atlanta 
International  Airport  was  modeled  to  establish  a  realistic 
operation  in  terms  of  procedures,  traffic  type  and  density,  and 
capacity  expectations.  An  airport  diagram  is  shown  in  the 
appendix.  In  order  to  do  a  preliminary  evaluation  of  the 
fidelity  of  the  simulation,  an  experienced  Atlanta  controller 
came  to  the  Technical  Center  to  examine  the  airspace  layout  and 
procedures.  Both  stimulus  (how  the  simulation  appears)  and 
response  (what  controllers  do  in  the  simulation)  fidelity  were 
considered  to  be  acceptable.  As  an  additional  check  on  fidelity, 
all  the  participants  were  to  be  interviewed  at  the  end  the 
experiment  and  asked  specifically  if  they  thought  the  simulation 
was  adequately  realistic. 


METHOD 


PARTICIPANTS. 

Personnel  involved  as  participants  in  this  study  were  qualified 
ATC  specialists  from  the  Atlanta  Terminal  Radar  Approach  Control 
Facility  (TRACON)  who  were  selected  from  a  group  of  approximately 
20  who  had  expressed  a  willingness  to  come  to  the  Technical 
Center  for  the  1  week  experiment.  Selection  of  the  participants 
was  made  by  the  Atlanta  TRACON  Operations  Office  based  on  the 
specifications  that  everyone  had  to  be  experienced  with  parallel 
approaches  and  that  they  should  present  a  range  of  overall 
experience.  Participants  were  current  in  approach  control 
procedures  and  had  worked  active  traffic  on  dependent  parallel 
runways  in  the  past  3  months.  Controllers  familiar  with 
dependent  parallel  operations  were  used  in  the  simulation  for  the 
following  reasons: 

1.  Training  time  and  practice  effects  should  be  minimal. 

2.  They  can  evaluate  the  realism  of  the  simulation. 

3.  They  are  better  able  to  evaluate  the  impact  of  any  changes  on 
their  own  ability  to  control  the  traffic  efficiently. 

Participants  had  to  be  physically  and  mentally  qualified  to 
perform  active  ATC  operations.  Due  to  the  relative  shortage  of 
full  performance  level  controllers,  participants  volunteered  on 
an  as  and  where  available  basis.  No  pretext  of  systematic 
sampling  is  made. 
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Participant  controllers  completed  an  entry  questionnaire  upon 
their  arrival  at  the  Technical  Center.  This  provided  an 
indicator  of  their  experience  and  current  attitudes.  The  four 
controllers  who  actually  worked  simulated  traffic,  ranged  in 
total  experience  from  7  years  3  months  to  19  years  (mean  12.6 
years).  They  had  spent  from  4  to  almost  15  (mean  6.0)  years, 
respectively,  at  Atlanta,  and  all  had  worked  parallel  approaches 
with  experience  ranging  from  2  years  10  months  up  to  16  (mean  7.8 
years) .  In  the  entry  questionnaire,  the  controllars  confirmed 
that  they  had  freely  volunteered  and  that  they  were  in  good 
health.  Three  out  of  the  four  stated  that  their  prior  stress 
level  was  low,  and  one  indicated  that  it  was  moderate.  The  two 
observers  were  both  experienced  controllers  and  supervisors. 

They  were  comfortable  with  making  over  the  shoulder  evaluations. 
Both  had  14  years  of  total  experience  and  a  mean  of  5  years 
working  parallel  approaches.  Atlanta  sent  a  very  qualified  and 
motivated  group  of  people  who  were  technically  competent  and 
highly  professional.  All  controllers  were  briefed  concerning 
their  rights  to  informed  consent  and  anonymity. 

QUALIFICATIONS. 

This  was  a  small  sample,  1-week  study  using  available  volunt: ers 
from  one  major  urban  tower  facility.  While  every  effort  was  made 
to  accomplish  as  much  as  was  scientifically  possible  with  the 
limited  number  of  controllers  available,  any  results  should  be 
viewed  as  indicative  rather  than  conclusive.  Subsequent 
decisions  concerning  changes  to  approach  separation  minima  should 
be  done  using  all  information  available,  including  expert 
judgement,  possible  replications  of  this  study,  and  old  fashioned 
common  sense. 

SIMULATION  FACILITY. 

This  study  was  accomplished  using  the  National  Airspace  System 
Simulation  Support  Facility  (NSSF) ,  which  is  an  ATC  simulator  at 
the  Federal  Aviation  Administration  (FAA)  Technical  Center, 
Atlantic  City  International  Airport,  New  Jersey.  The  NSSF  is  a 
general  purpose  ATC  simulator  designed  to  provide  a  realistic 
test  bed  for  developing,  testing,  and  evaluating  advanced  ATC 
concepts,  airspace  management  plans,  and  procedures.  The 
simulator  consists  of  three  subsystems:  ttv'  Controller 
Laboratory,  the  Simulator  Pilot  Complex,  and  the  Central  Computer 
Facility. 

The  Controller  Laboratory  is  a  simulated  en  route  or  terminal 
control  room  which  includes  eight  radar  displays  and  the 
associated  keyboard  entry  and  communication  equipment.  The 
laboratory  is  configured  so  that  the  participant  controllers  can 
function  in  a  manner  nearly  identical  to  the  way  they  do  in  the 
field.  Controller-to-controller ,  controller-to-pilot  (simulator 
operator) ,  and  pilot -to-controller  communications  are  available 
and  was  utilized  in  this  simulation.  The  controller  portion,  or 
subsystem,  provides  the  sights  and  sounds  of  the  ATC  control 
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room.  While  it  is  not  a  perfect  copy  of  the  radar  room  of  an 
approach  control  (stimulus  fidelity) ,  it  does  provide  fairly 
realistic  opportunities  for  controller  reactions  to  a  variety  of 
real  world  situations  (response  fidelity) . 

The  second  subsystem  of  the  NSSF  involves  people  who  serve  as  the 
"pilots"  of  the  aircraft  under  control.  These  simpilots  are  in 
voice  contact  with  the  controller  and  respond  to  his  directions. 
They  fly  their  computer  generated  aircraft  from  a  keyboard  in  an 
adjacent  room.  One  simpilot  controls  the  flight  of  up  to  10 
aircraft.  The  simpilots,  being  human  beings,  do  make  errors.  To 
a  certain  extent,  this  adds  to  the  realism  of  the  simulation. 
However,  when  they  make  errors  which  a  pilot  would  not  ordinarily 
make,  then  it  reduces  realism.  This  did  happen  on  occasion 
during  the  experiment  and  was  handled  either  by  the  controller 
who  recovered  the  situation  or  by  the  test  controller  who  removed 
the  specific  flight  track  (simulated  aircraft)  from  the  problem. 

The  final  subsystem  is  the  computer,  which  serves  as  both  a 
target  generator  and  as  the  collector  of  all  systems  information. 
This  computer,  a  Gould  SEL,  samples  the  simulated  airspace  every 
second  and  records  all  aircraft  information  to  be  described  in 
more  detail  under  a  latter  section  of  this  design. 

The  operation  of  the  simulation  facility  was  the  responsibility 
of  the  test  director.  He  coordinated  with  the  technicians, 
simulator  operators,  computer  operators,  and  other  personnel  and 
organizations  associated  with  the  test  effort. 


Each  simulation  run  consisted  of  two  separate  and  completely 
independent  airports.  Each  had  two  Final  Controller  positions, 
divided  into  north  and  south  arrivals.  Each  position  controlled 
its  own  runway.  Participant  controllers  were  each  assigned  a 
letter  code  from  A  to  D  which  was  used  to  schedule  their  activity 
and  served  as  an  identifier  on  all  documents  generated  by  the 
experiment.  Controllers  functioned  in  two-person  teams,  the 
composition  of  which  was  rotated.  There  were  six  possible 
combinations  of  two  controllers: 

Controller  Team  Combinations 

AB 

CD 

AD 

CB 

AC 

BD 

Rotation  of  personnel  served  several  purposes.  First  it  balanced 
out  any  potential  effects  based  on  the  interpersonal  "Chemistry" 
of  any  controller  pair.  Second,  given  that  we  can  treat  each 
pair  as  a  functional  unit  or  team,  then  we  have  effectively 


increased  the  sample  size  from  four  to  six,  providing  additional 
statistical  power  for  the  research  design.  This  appeared 
feasible  based  on  conversations  with  the  consultant  from  Atlanta, 
indicating  that  operating  dependent  parallel  approaches  is  a  very 
team  oriented  effort. 

The  primary  independent  variable  in  this  study  was  the  minimum 
diagonal  separation  allowed  between  aircraft  making  simultaneous 
approaches  to  Atlanta  runways  9R  and  9L.  This  variable  was  set 
at  two  levels:  the  current  2  nmi  separation  and  the  proposed 
distance  of  1.5  nmi.  The  basic  research  design  is  depicted 
graphically  in  the  attached  figure  1  labeled  "Research  Design." 
Every  participant  was  to  experience  three  iterations  at  each 
separation  distance  and  in  team  combination  with  each  of  the 
other  participants.  However,  due  to  unavoidable  computer 
failure,  almost  2  days  of  testing  were  lost  and  the  design  had  to 
be  amended.  Each  controller  team  participated  under  each 
condition  twice.  So  instead  of  18  hours  of  simulation,  12  hours 
were  completed.  Since  there  were  two  independent  airports  run 
simultaneously,  this  provided  the  equivalent  of  24  hours  of 
simulation.  Each  controller  rotated  not  only  team  membership, 
but  also  runway  assignment.  They  worked  runways  9R  and  9L  an 
equal  number  of  times. 

When  the  simulation  was  run,  the  targets  v«re  automatically 
started.  The  targets  contacted  the  arrival  sectors  immediately. 
However,  there  was  a  means  for  controllers  to  refuse  inbound 
traffic  if  they  felt  they  had  become  saturated.  This  was  to  be 
accomplished  by  the  simulation  manager  and  would  have  involved 
holds  or  elimination  of  aircraft  from  the  traffic  stream.  Using 
radar  vectors,  speed  control,  and  altitude  separation,  the  final 
controllers  were  to  land  his/her  aircraft  on  the  designated 
discrete  runway.  Runway  switching  at  team  discretion  was  allowed 
as  it  is  in  the  operational  facility. 

DESIGN  SUMMARY. 

The  four  controllers  were  systematically  established  as  two  teams 
of  two,  and  each  team  worked  together  for  a  1-hour  of  simulation; 
then  team  membership  was  rotated.  The  administration  of  this 
operation  is  described  in  the  attached  matrix  labeled  "Parallel 
Approaches  Administrative  Order"  (table  1) .  Also,  in  order  to 
show  that  each  team  was  exposed  to  each  condition  twice,  team 
membership  is  overlaid  on  the  research  design  in  the  chart 
labeled  "BASIC  RESEARCH  DESIGN"  (figure  2).  Each  team  was 
assigned  to  one  of  two  identical  configurations.  Each  run  then 
included  two  independent  operations  of  the  system;  that  is  to 
say,  two  independent  but  identical  simulations  going  on  at  the 
same  time.  Team  members  were  asked  to  switch  between  the  left 
and  right  runways  (north  and  south  sides  of  the  display) 
periodically  after  1-hour  runs. 
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FIGURE  1.  RESEARCH  DESIGN  FOR  PARALLEL  APPROACHES 
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Design  Summary 

1.  Six  team  combinations. 

2.  Two  replications  for 
each  team  under  each 
separation  condition. 

3 .  Twelve  record  runs 
preceeded  by  two 
training  runs. 

4 .  Order  of  letters  in  team 
membership  signifies  right 
and  left  runway  assignments 
respectively! 


FIGURE  2.  BASIC  RESEARCH  DESIGN 
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PROCEDURE. 


When  the  controllers  arrived,  they  were  briefed  on  the  background 
of  the  simulation  effort;  how  the  simulation  was  to  be  conducted 
and  what  was  expected  of  them.  Essentially,  the  controllers  were 
advised  to  function  as  they  normally  do.  They  were  also  asked  on 
several  occasions  to  take  all  the  separations  seriously  and  to 
run  as  many  aircraft  as  close  as  they  felt  comfortable.  They 
were  advised  that  they  would  be  given  a  questionnaire  after  every 
test  run  and  that  there  would  be  a  debriefing  at  the  conclusion 
of  their  test  participation.  The  purpose  of  the  debriefing  was 
to  solicit  feedback  from  the  controllers  on  the  overall 
simulation  and  any  areas  which  could  be  improved. 

The  progression  of  a  participant  controller  through  the 
experiment  is  described  graphically  in  figure  3.  After  an 
initial  welcome  and  description  of  project  goals  to  include  an 
informed  consent  briefing,  the  controller  was  asked  to  complete  a 
brief  questionnaire  describing  his/her  background  in  ATC  and 
current  motivation  for  this  project.  Once  entry  processing  was 
complete,  a  period  of  training  and  familiarization  began.  This 
training  was  based  on  the  instructional  systems  design  (ISD) 
model  which  calls  for  the  periodic  evaluation  of  progress  and  for 
feedback  of  the  results  of  that  evaluation  to  both  the  trainee 
and  to  the  training  system.  The  evaluation  was  primarily  based 
on  expert  judgement  of  in-house  observer/ evaluators.  The 
training  objectives  included  tasks,  conditions,  and  standards  as 
described  the  training  and  familiarization  plan  which  is  attached 
as  an  appendix  to  this  report.  At  the  end  of  the  first  few  hours 
of  training,  a  decision  point  was  reached  as  to  whether  or  not 
the  participants  were  ready  or  should  have  more  training.  Both 
the  test  director  and  all  the  participant  controllers  agreed  that 
they  were  ready  to  proceed  with  record  data  collection  at  this 
point. 

Prior  to  the  beginning  of  data  collection,  each  participant  was 
assigned  to  one  of  the  preselected  administrative  orders  of  the 
different  combinations  of  the  independent  variables.  All  data 
collection  was  accomplished  using  an  arbitrary  letter  code  pre¬ 
assigned  to  each  participant.  No  names  were  recorded  on  any 
forms  and  the  list  of  names  by  codes  maintained  exclusively  by 
the  experimenter  was  destroyed  at  the  conclusion  of  the 
experiment.  This  was  to  protect  the  privacy  of  the  participants 
and  to  encourage  their  openness  and  honesty  when  completing 
questionnaires  and  interviews. 

A  typical  data  collection  run  proceeded  as  follows.  Prior  to  the 
run  the  experimenter  informed  the  simulation  manager  of  the 
separation  distance  to  be  used  for  that  run.  The  simulation  was 
set  up  accordingly.  Participants  were  relatively  quick  to  move 
into  a  routine.  Each  wore  a  badge  with  his  letter  code  on  it, 
and  they  went  to  the  position  that  was  marked  with  their 
respective  letters  for  that  hour  of  simulation.  The  participant 
then  took  control  of  the  airspace  in  the  designated  position. 
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Each  data  run  lasted  approximately  1  hour  (run  7  was  4  minutes 
short  due  to  computer  failure)  and  involved  free  play  simulation 
in  which  the  participants  made  all  the  decisions  normally  made  by 
an  individual  in  his/her  position.  Data  collection  occurred  both 
during  and  after  each  simulation  run.  During  the  run  it 
consisted  of  both  manual  and  automated  methods.  The  manual 
system  was  based  on  the  continuous  observation  of  two 
observer/evaluators  who  made  entries  every  30  minutes  on  an 
evaluation  forms.  The  automated  system  involved  the  continuous 
sampling  by  the  simulation  itself  of  systems  variables  which 
included  aircraft  status,  changes  in  status,  separation  between 
aircraft  pairs,  and  participant  controller  actions.  The 
simulation  system  can  provide  these  data  in  raw  form  or  with  a 
considerable  amount  of  processing  to  include  accumulation  over 
time  intervals.  After  each  data  run,  the  participant  was  asked 
to  complete  a  questionnaire  (shown  in  the  appendix)  designed  to 
gauge  their  assessment  of  how  hard  they  had  to  work  on  that  run 
and  how  they  felt  that  they  performed.  Also  present  at  every  run 
was  an  Engineering  Research  Psychologist  (ERP)  -  the  author.  He 
took  observational  notes  and  recorded  any  incidents  which  may 
have  had  an  impact  on  the  data. 

SIMULATION  RUNS. 

Data  runs  were  designed  to  be  exactly  60  minutes  in  duration  with 
a  20-  to  30-minute  turnaround  between  runs.  Run  7  ended  4 
minutes  early  due  to  computer  failure.  Run  8  had  to  be  repeated 
when  it  also  ended  approximately  40  minutes  from  its  beginning. 
However,  there  was  a  time  lag  measured  in  days  before  the 
computer  came  up  again.  Wednesday  of  the  data  week  and  most  of 
Thursday  were  lost  before  the  system  became  operational.  Data 
collection  at  the  Technical  Center  was  scheduled  between  the 
hours  of  0830  and  1630  with  the  following  tentative  time  blocks: 

1.  0830-0930  2.  1000-1100  3.  1130-1230 

4.  1330-1430  5.  1500-1600 

On  Thursday,  the  system  became  operational  at  approximately  1630 
hours,  and  three  runs  were  completed  by  2000  hours,  at  which  time 
the  simpilots  had  to  be  released.  Participant  controllers 
volunteered  to  stay  late  and  they  did  eat  prior  to  the  first  run 
so  they  were  not  working  hungry. 

TRAFFIC  SAMPLES. 

The  mix  of  traffic  used  for  this  simulation  was  taken  directly 
from  the  actual  flight  strips  at  the  Atlanta  International 
Airport  Tower.  Traffic  samples  were  heavy  enough  to  allow  the 
maximum  system  capacity  to  be  reached.  Traffic  included  small 
light  and  heavy  aircraft  requiring  controllers  to  adjust 
longitudinal  separation  according  to  aircraft  mix. 
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Table  2  describes  the  separation  criteria  for  aircraft  in  trail 
and  also  specifies  the  critical  cutoffs  for  the  data  screening 
for  violations.  These  data  were  taken  directly  from  Atlanta 
Tower  sources. 


TABLE  2.  SEPARATION  CRITERIA 


Trailing  Leading  Separation 

Aircraft  Aircraft  Requirements  (nmi) 
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DATA  COT.T.EOTTON. 

The  data  in  this  experiment  were  available  from  a  number  of 
different  sources,  both  objective  and  subjective.  The  objective 
data  set  included  all  measurements  taken  and  recorded  by  the 
simulation  computer  during  each  hour  of  simulation.  This 
consists  of  virtually  everything  that  occurred  and  constitutes  a 
data  base  which  is  more  extensive  than  is  required  to  meet  the 
objectives  of  this  experiment.  Only  a  subset  was  to  be  analyzed 
in  order  to  try  and  answer  the  questions  posed  in  the  objectives 
of  the  project.  To  do  otherwise  would  have  delayed  the  analysis 
and  reporting  function  for  a  lengthy  period.  The  subjective  data 
included  participant  questionnaires  and  interviews  and  observer 
inputs  for  each  hour  of  simulation.  This  data  subset  was  focused 
for  the  purposes  of  this  specific  experiment,  and  as  such,  was 
analyzed  in  its  entirety.  Both  the  objective  and  subjective  data 
subsets  were  relatively  complete  having  practically  no  lost  data 
points.  Only  two  exceptions  existed.  Run  7  in  which  4  minutes 
or  6.6  percent  of  the  data  were  lost,  and  several  post-run 
questionnaires  in  which  the  participant  missed  a  response.  In 
the  latter  case  a  decision  was  made  to  use  an  estimator  (the  mean 
of  the  group)  for  the  missing  responses.  In  the  case  of  the 
missing  4  minutes,  the  decision  was  to  ignore  it  since  it  does 
not  seem  to  have  made  any  appreciable  difference. 
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RESULTS 


DATA  ANALYSIS. 


The  analysis  of  the  research  data  was  accomplished  along  two 
parallel  paths  each  depending  on  the  nature  of  the  data  being 
analyzed.  Information  collected  by  computer  consisted  of  a  large 
volume  of  numerical  data  that,  by  itself,  was  not  interpretable. 

It  had  to  be  reduced  using  routines  available  in  the  simulation 
computer.  The  reduction  process  produced  output  on  each  run  to 
include  details  of  every  airspace  violation.  The  process  also 
generated  summary  data  which  provided  tallies,  totals,  medians, 
means,  and  variability  statistics  on  a  host  of  measures,  many  of 
which  had  little  relevance  to  the  objectives  of  this  study.  Data 
on  each  hour  of  simulation  were  broken  out  by  individual 
controller  and  by  controller  teams.  However,  since  the  operation 
of  each  airport  was  primarily  a  team  effort  and  controllers 
running  staggered  approaches  normally  shift  aircraft  between  the 
parallel  runways  (depending  on  how  they  develop  their  shared 
strategies) ,  it  was  decided  to  focus  all  analytic  effort  on  team 
data  generated  through  the  simulation  itself.  Therefore,  the 
information  considered  essential  centered  on  both  success  in  the 
form  of  productivity  and  errors  defined  as  violations  of  the 
standards  in  force.  Team  data,  which  was  considered  significant 
in  terms  of  its  potential  to  answer  questions  posed  in  the 
objectives  of  the  test  plan,  was  transferred  to  spread  sheets  for 
further  analysis. 

Post-run  questionnaire  data  were  reduced  by  entering  the 
numerical  information  into  a  lotus  spreadsheet.  Descriptive 
statistics  were  computed  using  standard  Lotus  formulas  which  were 
resident  within  the  spreadsheet  itself;  observer  information  was 
handled  the  same  way.  However,  since  the  observers  generated  a 
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considerable  amount  of  conceivably  useful  qualitative  data  in  the 
form  of  written  comments,  it  was  decided  to  have  these  typed  in 
draft  form  for  reference  purposes.  The  controller  exit  interview 
was  handled  qualitatively  since  there  was  very  limited 
quantitative  information  on  it.  The  results  in  the  section  which 
follows  will  be  reported  in  blocks  based  on  the  source  of 
measurement.  Any  conclusions  drawn  from  the  results  and 
discussion  by  the  reader  should  be  made  in  light  of  a  full 
understanding  of  the  qualifications  expressed  earlier  in  this 
report. 

DATA  PRESENTATION  AND  STATISTICS. 

The  approach  taken  in  this  report  is  based  on  trying  to  answer 
the  questions  posed  in  the  test  plan  using  to  the  extent  possible 
simple  and  straight  forward  statistical  techniques.  Also,  it 
should  be  kept  in  mind  that  this  was  a  very  limited  sample  study 
and  any  generalizations  should  be  guarded.  The  statistical 
techniques  used  for  this  report  are  summarized  below. 

The  most  simple  and  common  techniques  are  referred  to  as 
descriptive  statistics.  These  explain  what  the  sample  looks  like 
in  terms  of  central  tendency  and  spread.  The  mean  or  arithmetic 
average  is  the  most  commonly  used  measure  of  central  tendency  and 
the  standard  deviation  is  the  most  familiar  indicator  of  spread 
or  distribution  around  the  mean.  Means  will  be  used  extensively 
and  especially  as  data  points  in  bar  graphs  to  try  and  make 
relationships,  or  the  lack  thereof,  jump  out  for  the  reader. 

Correlation  techniques  are  somewhat  more  complicated  and  include 
both  correlation  and  multiple  linear  regression.  Correlation 
examines  how  variables  are  related  to  each  other  and  evaluates 
this  covariation  in  terms  of  how  much  variability  (standard 
deviation)  exists  within  each  of  the  variables.  Correlations 
range  from  -1,  a  perfect  inverse  relationship,  to  +1,  a  perfect 
positive  relationship.  The  closer  one  approaches  to  either  +1  or 
-1,  the  more  knowing  about  one  variable  (i.e.,  working 
conditions)  tells  you  about  another  variable  (i.e.,  operator 
workload)  in  your  sample.  Multiple  linear  regression  is  an 
outgrowth  of  correlation.  It  provides  a  means  of  comparing  the 
relationship  of  multiple  independent  variables  such  as  post-run 
questionnaire  items  against  a  dependent  variable  like  run 
sequence  or  separation  minimums. 

The  resultant  product  is  a  weighted  linear  sum  of  the  independent 
variables  which  maximizes  their  ability  to  account  for 
variability  in  the  dependent  variable.  The  computed  weights 
indicate  the  degree  to  which  each  of  the  variables  or  measures 
contributes  to  the  strength  of  the  relationship. 

The  last  statistical  technique  to  be  described  is  the  analysis  of 
variance  or  ANOVA.  To  the  extent  that  we  can  and  where  the 
quality  of  the  data  permits,  it  is  desirable  to  try  and  make 
inferences  about  the  world  outside  of  the  laboratory — the 
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population  from  which  the  sample  was  drawn.  All  inferential 
tests,  including  ANOVA,  take  into  consideration  two  kinds  of 
variance:  treatment  and  error.  Treatment  variance  is  the  result 
of  the  experimental  manipulation  of  independent  variables.  In 
the  current  experiment  there  are  two  independent  variables:  the 
separation  minima  and  the  replication.  We  need  to  know  the 
degree  to  which  differences  in  sample  data  are  the  result  of  our 
independent  variables  and  the  degree  to  which  the  results  might 
have  occurred  by  chance  or  error.  Error  results  from  variance 
within  the  sample  or,  in  this  particular  case,  the  differences  in 
performance  across  our  six  controller  teams.  The  ANOVA  used  in 
this  project  will  evaluate  the  effects  of  changing  the  separation 
minima  between  2.0  and  1.5  nrni,  effects  of  replication,  and  the 
interaction  between  the  two  independent  variables.  Ideally, 
there  would  be  main  effects  for  separation  and  none  for 
replication  or  interaction.  Replication  main  effects  would 
indicate  a  learning  or  habituation  with  experience  which  is 
unrelated  to  our  major  purpose.  An  interaction  between 
separation  and  replication  complicates  interpretation 
considerably  and  means  that  neither  can  be  explained  independent 
of  the  other.  A  statistical  interaction  between  two  variables 
implies  that  they  somehow  influence  each  other.  In  order  to 
understand  the  results  from  each  variable  (i.e.,  the  impact  of 
separation  distance  on  violation  frequency)  you  must  separate  out 
the  effects  of  the  other  variable  (i.e.,  the  level  of 
replication) .  If,  for  example,  controllers  are  changing  their 
behavior  with  experience  in  the  simulation,  then  each  replication 
would  have  to  be  treated  separately.  The  results  of  ANOVA  are 
reported  as  "F"  values  which  range  from  0  and  up.  The  computed 
values  are  compared  against  a  probability  table,  an  F 
distribution,  to  determine  the  likelihood  that  the  computed  F 
occurred  by  chance.  By  convention,  those  F's  which  may  have 
occurred  by  chance  either  1  percent  (P<.01)  or  5  percent  (P<.05) 
or  less  are  considered  significant,  and  generalizing  to  the 
population,  taking  other  limitations  into  account,  is  considered 
reasonable. 

The  results  in  terms  of  data  and  statistical  analysis  will  be 
presented  in  the  following  sections.  First  will  be  the  so  called 
hard  data  which  were  gathered  by  the  computer  during  each 
simulation  run.  These  data  pertain  to  productivity  and 
performance  of  the  controller  teams  working  under  the  two 
separation  conditions.  Second  will  be  the  data  generated  by 
observers  and  the  post-run  questionnaire.  These  data  touch  on 
performance,  but  also  provide  estimates  of  workload  from  two 
perspectives,  the  controllers  themselves  and  the  observers. 

SYSTEM  DATA. 

Table  3  summarizes  the  means  and  standard  deviations  for  six  key 
variables  which  are  being  used  to  estimate  controller  performance 
and  productivity.  Recalling  that  the  research  design  was 
developed  to  evaluate  any  differences  induced  by  the  two 
separation  standards  and  also  to  investigate  the  possibility  of 


15 


(Standard  deviations  in  parentheses) 


S 

e 

P 

a 

r 

a 

t 

1 

o 

n 


S 

e 


s 

e 

P 

a 

r 

a 

t 

i 

o 

n 


Longitudinal 

Violations 

c 

Longitudinal 

Durations 

* 

1 . 5 

7 . 17 
(3.97)' 

5 . 67 
(4.19) 

i 

6.42 

o 

e 

p 

a 

1.5 

153.26 

(52.53) 

1 

156.69 

(56.70) 

154 . 97 

2.0 

3.83 

(3.71) 

6 . 17 
(2.60) 

5.00 

I 

a 

t 

i 

2 . 0 

145.92 

(39.60) 

i 

139.80 

(53.67) 

142.86 

5 . 50 

5.90 

Q 

n 

149 . 58 

148 . 25 

R1  R2 

Replication 

R1  R2 

Replication 

Parallel 

Violations 

c 

Parallel 

Durations 

* 

1 . 5 

13.33 

(4.57) 

12 . 50 
(4.82) 

12.92 

o 

e 

p 

a 

1 . 5 

66 . 25 
(11.65) 

52 . 37 
(5.77) 

59.31 

2 . 0 

27 . 00 
(8.16) 

34.17 

(4.41) 

30.58 

r 

a 

t 

i 

2.0 

74 . 55 
(16.13) 

61.45 

(15.32) 

67.99 

20.17 

23.33 

o 

n 

70.40 

56.91 

R1  R2 

Replication 

R1  R2 

Replication 

Aircraft 

Handled 

c 

Aircraft 

Landed 

1 .  5 

95 . 67 
(2.36) 

93 . 50 
(3.91) 

94 . 58 

o 

e 

P 

a 

1 .  5 

67 . 83 
(3.02) 

68.17 

(5.18) 

68.00 

2.0 

88.00 

(4.9) 

96.67 

(2.05) 

92 . 33 

r 

a 

t 

i 

2 . 0 

61 . 67 
(5.62) 

67 . 00 
(4.62) 

64.33 

91.83 

95.08 

0 

n 

64.75 

67 . 58 

R1  R2 

Replication 


R1  R2 

Replication 


■’Note:  Duration  data  is  computed  by  dividing  total  time  of  all 
violations  per  hour  of  simulation  by  the  number  of  vio¬ 
lations  for  that  hour. 
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replication  effects  which  is  an  undesirable  outcome  from 
participant  learning  or  fatigue  (to  mention  a  few  possibilities) . 
The  first  four  variables  are  measures  of  controller  errors  in 
terms  of  violations  of  airspace  minima.  Every  time  a  simulated 
aircraft  approaches  another  aircraft  at  less  than  the  minimum 
distance,  the  computer  tallies  the  event  and  begins  counting  time 
in  seconds  until  the  violation  ends  or  the  aircraft  xand.  As  we 
examine  the  information  described  in  table  3  there  are  a  number 
of  things  to  keep  in  mind.  The  information  presented  is 
descriptive  and  differences  which  exist,  or  appear  to  exist, 
between  the  means  may  well  be  a  function  of  sampling  error  and 
not  be  significant  when  the  error  within  the  sample  is  taken  into 
account.  If  the  differences  between  means  is  relatively  large  as 
compared  to  their  respective  standard  deviations,  then  the 
chances  are  increased  that  we  may  be  seeing  something  of 
signif icance.  Another  method  when  examining  mean  summary  tables 
is  to  look  for  relative  differences  across  the  variables.  In 
this  design,  examine  the  shift  in  the  means  for  the  two 
separation  distances  for  the  two  replications.  If  it  is  as 
different  as  it  is  for  the  longitudinal  violation  frequencies, 
there  may  be  an  interaction  which  will  complicate  interpretation 
considerably. 

Another  way  of  presenting  the  violation  data  is  seen  in  the  four 
bar  graphs  in  figures  4  through  7.  These  graphs  provide  a  first 
quick  look  at  the  data  and  at  the  relationships  across  the  levels 
of  the  two  variables.  For  example,  there  appears  in  figure  4 
that  there  may  have  been  an  increase  in  longitudinal  violations 
across  the  replications  for  the  2  nmi  separation  standard  but  not 
for  the  1.5  nmi  standard.  While  this  will  turn  out  to  be  true  in 
this  case,  as  will  be  seen  shortly,  you  can  only  go  so  far  with 
graphical  analysis  and  should  do  so  with  a  great  deal  of  caution. 
The  next  step  in  the  analysis  is  to  employ  more  powerful 
statistical  techniques. 

Analysis  of  Variance  (ANOVA)  was  used  extensively  on  the  systems 
variables  including  violation  frequencies,  durations,  and 
controller  productivity  measure  of  aircraft  handled  and  landed. 
This  amounted  to  applying  ANOVA  to  each  of  the  variables 
summarized  in  table  3.  The  first  variable  to  be  analyzed  was 
longitudinal  conflicts.  A  two-way  ANOVA  separation  minimum  by 
replication  indicated  a  significant  interaction  (F=14.29,  P<.05, 
df ,  5) .  As  indicated  earlier,  an  interaction  complicates 
interpretation  and  both  variables  have  to  be  taken  into  account. 
Figure  8  plots  the  mean  frequencies  of  the  longitudinal 
violations.  Table  4  following  represents  the  simple  main  effects 
for  all  the  combinations  of  interest. 

The  differences  between  the  means  for  the  first  replication  are 
significant.  There  were  more  violations  when  the  standard  was 
1.5  nmi  than  when  it  was  set  at  2.0  nmi.  This  difference  washed 
out  over  the  two  replications  making  its  relevance  somewhat 
questionable.  There  was  a  significant  increase  in  violations  as 
controllers  proceeded  across  the  two  replications  using  the  2  nmi 
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FIGURE  5.  LONGITUDINAL  CONFLICT  DURATIONS 
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FIGURE  8.  LONGITUDINAL  VIOLATION  INTERACTION  GRAPH 


standard.  By  the  time  of  the  second  replication  there  was  no 
real  difference  between  the  two  standards  in  terms  of 
longitudinal  violations.  This  may  have  been  a  function  of 
overconfidence  or  fatigue  on  the  part  of  the  controllers  when  the 
2  nmi  standard  was  in  force. 


TABLE  4.  LONGITUDINAL  VIOLATIONS  SIMPLE  MAIN  EFFECTS 


Effect 

F  V31U3 

Probability 

Separation  at  Replication  1 

21.61* 

<  .01 

Separation  at  Replication  2 

.07 

>  .05 

Replication  at  2  nmi 

10.57** 

<  .05 

Replication  at  l.o  nmi 

4.37 

>  .05 

*p<.01  **p< . 05 

The  results  for  the  mean  durations  of  longitudinal  conflicts  must 
be  interpreted  not  only  from  an  examination  of  the  means 
themselves  (figure  5) ,  but  also  from  taking  into  consideration 
the  variability  within  each  cell  or  condition.  An  ANOVA 
indicated  no  interaction  between  the  durations  for  the  two 
separation  standards  and  the  replications  (F-.026,  P>.05,  df 
1,5).  There  was  also  no  main  effect  for  separation  standard 
(F-1.43,  P>. 05,  df  1,5)  and  no  effect  from  the  replication 
(F-0.004,  P>. 05,  df  1,5).  In  other  words,  there  was  no 
significant  impact  of  the  independent  variables  on  longitudinal 
violation  durations.  While  some  impact  might  have  been  assumed 
by  examining  the  mean  summary  table  or  the  bar  graph,  the 
internal  variability  estimated  by  the  standard  deviations  was 
relatively  high  and  eliminated  the  probability  of  significance. 

The  findings  for  parallel  violation  frequencies  are  probably  the 
cleanest  results  in  the  entire  experiment.  An  examination  of 
figure  6  indicates  the  possibility  of  considerably  fewer 
violations  when  using  the  1.5  nmi  standard.  A  two-way  ANOVA 
produced  no  significant  interaction  (F-5.65,  P>.05,  df  1,5) 
between  the  two  key  variables  and  there  was  no  effect  from 
replication  (F-2.81,  P>.05,  df  1,5).  There  was,  however,  a  very 
strong  effect  based  on  the  separation  standard  employed 
CF*139.06,  Pc.001,  df  1,5).  The  probability  of  this  effect 
occurring  by  chance  due  to  sampling  error  is  less  than  1  in  1000. 
Significantly  fewer  parallel  separation  violations  occurred  when 
the  separation  standard  in  force  was  1.5  nmi  than  when  2.0  nmi 
was  used.  The  results  for  the  mean  durations  of  parallel 
violations  were  the  same  (figure  7).  There  was  no  interaction 
(F*. 009 ,  P> . 05 ,  df  1,5)  and  there  was  no  effect  resulting  from 
replication  (F-1.77,  P>.05,  df  1,5).  The  difference  between  the 
mean  durations  of  59.31  seconds  and  67.99  seconds  for  the 
standards  of  1.5  and  2.0  nmi,  respectively,  was  significant 
(F-6.92,  P< . 05 ,  df  1,5) . 
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Longitudinal  violations  must  be  determined  based  in  part  on  the 
nature  of  the  aircraft  involved  to  take  into  consideration  the 
wake  vortex  turbulence.  This  means  that  the  distances  vary  for 
an  official  violation  using  the  criteria  described  back  in 
table  2.  The  parallel  violations  were  assessed  based  on  the 
absolute  standards  of  2  or  1.5  nmi.  Given  this,  one  might  ask 
just  how  close  aircraft  actually  approached  each  other  when  they 
were  as  close  as  they  were  going  to  become  during  a  parallel 
violation.  It  was  possible  to  tally  this  information  so  that  the 
distributions  of  violations  could  be  examined.  The  reader  may 
wish  to  examine  table  A-l  in  the  appendix.  The  table  describes 
the  distributions  of  violations  based  on  the  distance  of  aircraft 
from  each  other  at  the  point  of  greatest  risk.  There  is  little 
doubt  that  the  participants  ran  the  aircraft  "tightly, "  and  as 
one  controller  confided,  perhaps  "tighter  in  simulation  than  they 
normally  would."  In  no  case  did  either  separation  standard 
result  in  two  aircraft  ever  occupying  the  same  point  in  simulated 
airspace.  One  might  ask  whether  the  violations  that  were  "close" 
were  any  closer  for  one  separation  standard  or  the  other.  It  was 
decided  to  graph  the  violation  frequencies  for  the  subset  of 
parallel  violations  that  were  within  l  nmi  of  each  other, 
irrespective  of  altitude.  These  violations  were  said  to  be 
within  a  warm  window.  They  are  described  in  figure  9,  "Warm 
Window  Violations."  It  appears  (especially  given  run  8)  that  the 
1.5  nmi  runs  have  more  than  their  share.  However,  there  was  also 
run  10,  a  1.5  nmi  exercise  with  no  warm  window  violations.  Using 
an  analytic  technique  similar  to  ANOVA  called  a  T  test  for 
correlated  data,  it  was  determined  that  there  was  no 
statistically  significant  difference  in  the  frequencies  of  warm 
window  violations  across  the  two  separation  standards  (T^l.64. 
P>,95.  df  5) , 

One  question  which  would  arise  sooner  or  later  regarding  the 
parallel  violations  is  whether  or  not  the  significant  decrease  in 
violations  represents  some  change  in  controller  behavior.  The 
other  alterative  is  that  the  change  in  standard  has  primarily  a 
statistical  impact  simply  legitimatizing  many  situations  which 
would  have  been  violations  using  the  2  nmi  standard.  In  an 
effort  to  understand  this,  a  reanalysis  was  conducted  of  all  the 
parallel  conflict  data  for  the  even  numbered  runs,  those 
conducted  using  the  1.5  nmi  standard.  The  reanalysis  employed 
the  2  nmi  filter.  The  results  are  presented  on  the  right  side  of 
figure  10  with  the  left  side  being  the  same  data  as  in  figure  6 
for  comparison  purposes.  The  data  confirm  that  the  controllers 
were  packing  in  as  many  aircraft  as  they  could  handle.  They  were 
certainly  following  instructions  to  push  the  edge  of  their 
performance  envelope.  It  must  be  remembered  that  the  data  on  the 
right  side  represent  what  would  have  been  violations  if  the 
standard  had  been  2  nmi,  not  actual  violations  given  the  rules  in 
force  during  the  even  numbered  runs. 

The  principle  measure  of  productivity  collected  during  this 
experiment  was  the  frequency  of  aircraft  landed  during  the 
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FIGURE  9.  WARM  WINDOW  VIOLATIONS 
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FIGURE  10.  APPLICATION  OF  THE  2  NMI  STANDARD  TO  1.5  NMI  RUNS 


1-hour  time  blocks  in  which  the  simulation  was  run.  Figure  11 
presents  the  mean  frequencies  for  separation  and  replications; 
figure  12  provides  the  information  on  the  controller  teams.  An 
ANOVA  was  computed  on  the  landing  frequency  data.  There  was  no 
interaction  between  the  separation  and  replication  variables 
(F=1.14,  P>. 05,  df  1,5);  also,  there  was  no  main  effect  from 
replication  (F-1.51,  P>.05,  df  1,5).  There  was,  however,  a 
significant  main  effect  between  the  means  of  64.33  aircraft 
landed  using  2  nmi  separation  and  68.00  aircraft  landed  using  the 
1.5  nmi  standard  (F=9.87,  P<.05,  df  1,5).  Controllers,  on  the 
average,  were  able  to  land  5.7 percent  more  aircraft  using  the 
1.5  nmi  standard  than  under  conditions  when  the  2.0  nmi  standard 
was  in  force.  The  difference  in  landing  frequencies  under  the 
two  separation  conditions  may  have  actually  been  an  underestimate 
due  to  a  design  problem  with  the  traffic  samples  used  in  the 
experiment.  There  were  six  samples  which  were  rotated  across  the 
simulation  runs.  Inadvertently,  two  of  these  samples  contained  a 
heavier  traffic  load  which  would  favor  a  higher  landing 
frequency.  However,  the  majority  of  these  heavier  loads  occurred 
in  the  2.0  nmi  runs  which  worked  against  the  hypothesis  that 
there  would  be  an  improved  capacity  using  the  1.5  nmi  separation. 
Despite  this  complication,  the  1.5  nmi  runs  still  averaged  more 
aircraft  landed,  attesting  to  the  strength  of  the  effect. 

SAEET X- 

The  first  question  raised  with  any  proposed  change  to  operational 
procedures  is;  Does  the  change  pose  an  increased  risk  for  those 
involved?  Is  it  safe?  While  it  is  not  feasible  to  make  sweeping 
generalizations  from  a  small  sample  simulation  study,  the  results 
can  be  viewed  as  an  indicator  of  the  possibilities  in  the  real 
airspace.  A  very  useful  tool  was  developed  and  reported  by  Paul, 
Shochet,  and  Algoe  (1989) .  The  aircraft  proximity  index  (API) 
was  designed  to  provide  Ma  measure  of  the  seriousness  of  a  near 
miss  between  two  aircraft. *  The  measure  is  computed  when  two 
aircraft  are  in  an  airspace  violation  and  have  reached  their 
point  of  closest  approach.  A  detailed  explanation  drawn  directly 
from  the  Paul,  et  al.  (1989)  technical  report  is  presented  in  the 
appendix  of  this  report. 

The  possible  range  of  the  API  is  from  0,  which  is  the  absence  of 
a  technical  conflict,  to  100,  where  two  aircraft  attempt  to 
occupy  the  same  piece  of  airspace  at  the  same  point  in  time,  a 
midair  collision.  The  API  is  a  dynamic  value  which  can  be 
sampled  over  time  from  the  beginning  of  a  violation  until  it 
becomes  most  serious.  For  the  purpose  of  this  study,  the  API 
will  be  examined  at  the  point  of  closest  proximity. 

Tables  5  and  6  summarize  the  API  data  for  all  12  data  runs.  An 
examination  of  the  parallel  conflict  API  indicates  that  means  for 
the  two  separation  standards  do  not  appear  to  differ  to  any  great 
extent.  In  fact  the  grand  means  for  all  the  2  nmi  and  1.5  nmi 
simulation  runs  were  2.10  and  2.02,  respectively.  There  was  no 


27 


61.67 


Mean  Frequency  of  Landings 


28 


TABLE  5.  PARALLEL  CONFLICT  API  SUMMARY  DATA 


Separation  at 
Max  API 
H  V 


Run 

Seoaration 

Mean 

Mode 

Min 

Max 

(nmi) 

Iftj 

SD 

1 

2 

2.47 

1 

1 

18 

1.15 

6 

2.90 

3 

2 

1.60 

1 

1 

13 

1.28 

6 

1.89 

5 

2 

1.56 

1 

1 

5 

1.55 

2 

1.13 

7 

2 

1.26 

1 

1 

4 

1.25 

485 

0.64 

9 

2 

2.43 

1 

1 

23 

1.03 

2 

3.32 

11 

2 

3.29 

1 

1 

39 

.67 

62 

5.52 

2 

1.5 

2.33 

1 

1 

17 

.86 

23 

3.40 

4 

1.5 

2.54 

1 

1 

13 

.62 

377 

3.36 

6 

1.5 

1.44 

1 

1 

10 

.94 

163 

1.76 

8 

1.5 

2.38 

1 

1 

13 

.97 

1 

2.70 

10 

1.5 

1.00 

1 

1 

1 

1.36 

2 

0.00 

12 

1.5 

2.47 

1 

1 

13 

.95 

2 

3.16 

Note: 

H  =  height 
V  *  vertical 
SD  =  standard  direction 


TABLE  6. 

LONGITUDINAL  CONFLICT 

API 

SUMMARY 

DATA 

H 

V 

Run 

separation 

Mean 

Mode 

Mir. 

Max 

Lmil 

im 

SB 

1 

2 

1.57 

1 

1 

3 

3.81 

315 

0.85 

3 

2 

1.00 

1 

l 

1 

4.64 

0 

0.00 

5 

2 

1.36 

1 

1 

6 

2.28 

1 

1.33 

7 

2 

1.60 

1 

l 

4 

3.96 

1 

1.34 

9 

2 

2.55 

1 

1 

10 

2.06 

6 

3.45 

11 

2 

1.08 

1 

l 

2 

2.54 

6 

0.28 

2 

1.5 

2.20 

1 

1 

10 

2.04 

0 

2.82 

4 

1.5 

1.63 

1 

l 

5 

2.30 

0 

1.41 

6 

1.5 

1.31 

1 

l 

3 

2.46 

6 

0.60 

8 

1.5 

1.41 

1 

1 

4 

2.37 

0 

0.87 

10 

1.5 

4.00 

1 

1 

18 

1.72 

5 

6.87 

12 

1.5 

2.55 

1 

1 

9 

2.12 

0 

2.52 

Note: 

H  »  height 
V  *  vertical 
SD  *  standard  direction 
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significant  difference  in  mean  API  scores  for  parallel  violations 
between  the  two  separation  standards  (F-.033,  P>.05,  df  1,10). 
This  can  also  be  seen  in  figure  13,  which  graphically  depicts  the 
mean  parallel  violation  API  scores.  The  most  frequent  score  or 
mode  under  all  conditions  was  "l,"  indicating  a  violation  of 
minimal  severity.  For  parallel  violations  the  highest  API  scores 
for  2  nmi  and  1.5  nmi  were  39  and  17.  Translated  into  actual 
distancer  that  aircraft  were  from  each  other,  this  represented 
the  following  separations.  For  2  nmi  and  an  API  of  39,  the 
aircraft  were  separated  by  0.67  nmi  horizontal  and  62  feet 
vertical  distance.  For  1.5  nmi  the  aircraft  were  separated  by 
0.86  nmi  horizontal  and  23  feet  vertical  distance.  This  was  the 
worst  case  for  a  parallel  violation. 

The  longitudinal  violations  presented  a  very  similar  picture. 

The  grand  API  means  for  the  2  nmi  and  1.5  nmi  were  1.53  and  2.18. 
These  were  not  significantly  different  (F»  1.94,  P>.05,  df  1,10). 
Again,  the  mode  or  most  frequently  seen  API  on  any  given 
violation  was  "1,"  a  minimal  violation.  Figure  14  describes  the 
mean  API  scores  for  longitudinal  violations  across  the  12  runs. 
There  were  no  noteworthy  patterns .  The  two  most  extreme  API ' s 
were  10  and  18  which  represented  the  following  actual 
separations.  For  the  2  nmi  and  an  API  of  10,  aircraft  were 
separated  by  2.06  nmi  horizontally  and  6  feet  vertically.  For 
1.5  nmi  separation  and  an  API  of  18,  aircraft  were  separated  by 
1.7  nmi  horizontally  and  5  feet  vertically.  The  reader  is 
reminded  that  despite  the  two  separation  standards  that  formed 
the  focus  of  the  this  study  of  parallel  separations,  the 
traditional  standards  for  longitudinal  separation  as  described  in 
table  2  remained  in  force. 

While  caution  should  always  be  used  in  interpreting  the  lack  of  a 
statistically  significant  difference,  there  appeared  to  be  no 
evidence  that  reducing  the  parallel  separation  standard  to  1.5 
nmi  reduced  safety  in  any  way.  The  results  of  the  API  are  in 
agreement  with  the  data  collected  from  the  controllers  themselves 
as  will  be  seen  in  a  later  section. 

OBSERVER  DATA. 

The  two  observers  were  present  during  every  hour  of  simulation 
and  each  worked  at  the  same  position  during  the  entire 
experiment.  Controller  teams  rotated  through  the  positions  every 
hour  of  operation.  The  observers  were  asked  to  make  general 
observations  of  anything  of  interest  that  they  saw  on  a 
continuing  basis.  They  were  also  asked  to  make  workload  and 
performance  estimates  every  half  hour;  such  that  two  estimates 
were  made  for  each  hour  of  simulation.  The  air  traffic  scenarios 
were  designed  so,  that  the  traffic  and  the  task  load  would  build 
up  over  the  first  quarter  hour  or  so  and  it  was  believed  that  two 
estimates  would  be  more  accurate  than  one  while  helping  to  keep 
the  observers  focused.  There  was  not  any  way,  however,  to 
estimate  the  reliability  of  the  observers  because  each  was 
observing  a  different  set  of  controllers.  Without  two 
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FIGURE  14.  LONGITUDINAL  CONFLICT  MEAN  API  SCORES 


controllers  observing  on  each  position,  so  that  inter-rater 
reliability  could  have  been  computed,  the  observer  data  should  be 
considered  as  only  a  rough  cut  at  the  questions  under 
consideration . 

Observer  workload  estimates  are  plotted  for  the  12  runs  in  figure 
15.  An  examination  of  these  graphs  indicates  little,  if  any, 
systematic  relationship  between  observer  estimates  of  workload 
and  separation  standard.  The  basic  Pearson  correlation  between 
the  two  variables  was  r=.12  which  signifies  a  very  weak  positive 
relationship  indicating  a  very  slight  tendency  for  the  2  nmi  runs 
to  be  rated  as  higher  in  workload  (see  table  7  for  a  summary  of 
all  the  intar-correlations  in  this  section) .  This  may  be 
explained  in  part  by  the  tendency  of  observer  2,  in  particular, 
to  rate  the  2  nmi  separations  as  higher  workload  than  the  1 . 5  nmi 
runs.  This  did  not  mean  that  one  observer  was  more  reliable  than 
the  other,  only  that  they  were  likely  using  a  different  view 
point.  The  results  indicate  that  the  observers  were  doing  their 
best  to  make  their  estimates,  and  that  they  were  treating  each 
run  independently. 

There  is  no  evidence  from  the  observer  data  that  altering  the 
separation  standard  influences  estimated  workload  in  any 
systematic  way.  The  estimates  seem  to  be  more  closely  associated 
with  the  immediate  situation  to  include  the  controller  team 
members  and  how  they  establish  their  work  pattern.  There  is 
little  else  in  the  system  other  than  observer  variability  itself 
to  explain  the  pattern  of  workload  estimates.  For  the  most  part, 
they  remain  moderate  in  magnitude  and  in  a  range  most  controllers 
would  accept  without  complaint.  This  will  latter  be  verified  by 
post-experiment  controller  interviews. 

Observer  performance  estimates  over  the  12  runs  are  presented 
graphically  in  figure  16.  There  was  no  systematic  difference 
between  the  two  observers  on  these  performance  ratings  (F*3.56, 
P>.05,df  1,46).  There  was  a  small  positive  correlation  r=.20 
(table  7)  between  performance  and  run  sequence  number  implying  an 
improvement  over  time. 

Observer  ratings  of  performance  like  those  of  workload  indicate  a 
willingness  to  look  at  what  the  controllers  were  actually  doing 
and  making  an  honest  attempt  to  reflect  it  on  paper.  The  ratings 
of  performance  and  workload  were  inversely  correlated  r»-.36 
(table  7)  which  is  a  finding  that  replicates  the  results  of  other 
studies  done  with  both  air  crew  and  air  traffic  controllers  at 
the  Technical  Center  (see  Stein,  1984;  Stein,  1985).  A 
multilinear  regression  was  accomplished  using  separation  standard 
as  the  dependent  variable  and  the  two  observer  variables  as  the 
predictors  or  independent  variables.  This  analysis  produced  a 
multiple  R  squared  of  .08  which  meant  that  observers  were  not 
seeing  systematic  differences  in  workload  or  performance  based  on 
separation  standard.  This  was  confirmed  by  an  ANOVA  on  the 
regression  (F-1.96,  P>.05,  df  2,45)  which  was  not  significant. 
This  seems  somewhat  different  from  what  was  discovered  in  the 
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Observer  Mean  Workload  Estimates 
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FIGURE  15.  OBSERVER  WORKLOAD  ESTIMATES 


TABLE  7.  CORRELATIONS  OF  SUBJECTIVE  VARIABLES 
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FIGURE  16.  OBSERVER  PERFORMANCE  ESTIMATES 


system  data.  It  is  not,  however,  inconsistent  because  human 
observers  can  only  attend  to  and  evaluate  a  finite  amount  of 
information.  The  conventional  method  of  evaluating  controllers 
in  the  field  is  based  on  supervisor  evaluation,  and  despite  its 
limitations,  it  works  in  the  real  world.  The  observers  in  this 
experiment  saw  no  systematic  differences  in  performance  that 
could  be  linked  to  separation  standard. 


OBSERVER  O 


C£. 


Both  observers  wrote  down  their  observations  during  each  run  of 
the  simulation.  Their  observational  style  differed  in  that 
observer  1  wrote  while  on  position  directly  behind  the 
controllers  and  observer  2  moved  away  from  the  position  when  it 
was  felt  necessary  to  write.  Both  observers  took  their  roles 
very  seriously,  as  already  indicated  by  the  numerical  ratings 
they  assigned.  Their  verbal  comments  put  those  ratings  into 
context  and  helped  us  understand  what  it  was  they  were  looking 
for  when  they  assigned  numbers.  It  is  interesting  to  follow  the 
track  of  the  workload  and  performance  estimates  in  figures  15  and 
16,  respectively,  while  reading  the  related  comments  of  the  two 
observers.  It  was  decided  to  include  the  text  of  these  comments 
in  the  appendix  for  the  use  of  the  interested  reader.  This 
material  was  transcribed  and  was  edited  for  spelling  only.  The 
grammar  and  technical  jargon  were  retained  in  their  natural  state 
in  order  to  convey  the  same  meaning  that  the  observers  intended. 
Observer  comments  are  coded  by  a  six-digit  label.  The  first  two 
positions  are  the  observers  number  01  or  02.  The  second  two 
positions  represent  the  run  numbers  from  1  to  12.  The  last  two 
alpha  characters  designate  the  members  of  the  controller  team 
that  was  observed  for  that  hour.  Some  key  points  are  summarized 
in  the  following  paragraph. 

Both  observers  noted  the  importance  of  the  working  relationship 
between  the  members  of  the  controller  teams.  The  strategy  that 
they  established  and  their  flexibility  of  implementation  were 
critical  to  their  ability  to  perform  regardless  of  separation 
standard.  The  process  involved  the  establishment  by  mutual 
consent  of  a  game  plan  which  required  ongoing  negotiation  of  key 
parameters  such  as  airspeeds  and  aircraft  to  be  inserted  into 
gaps.  Once  a  workable  operation  was  running  smoothly,  the  amount 
of  verbal  coordination  was  reduced.  Controllers  could  cue  on 
each  other's  behavior  at  least  until  something  went  wrong.  The 
latter  category  included  pilot  errors,  occasional  system 
problems,  and  misjudgments  by  the  controllers  themselves. 
Coordination  increased  at  that  time  to  try  and  effect  a  recovery 
of  the  situation.  Observers  made  very  few  comments  relative  to 
the  success  or  failure  of  separation  maintenance.  In  run  5, 
where  the  estimated  workload  was  very  high,  both  observers 
described  the  nature  and  extent  of  the  problems  which  occurred. 
Observer  1  specifically  noted  that  the  stagger  or  overall 
parallel  separation  was  lost  after  a  sequence  of  pilot  errors 
threw  the  controllers'  game  plan  into  turmoil.  Both  observers 
appear  to  have  been  looking  for  the  smoothness  of  the  working 
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relationships  and  the  maintenance  of  some  semblance  of  order  in 
the  airspace.  Observer  2  commenting  on  run  2,  indicated  that 
there  appeared  to  be  no  difference  in  technique  or  turn  on  (...to 
final)  separation  for  the  two  separation  standards.  The 
observers  were  our  subject  matter  experts  during  the  experiment. 
Their  comments,  while  subjective,  can  serve  as  a  frame  of 
reference  for  the  interpretation  of  the  rest  of  the  data. 

POST-RUN  QUESTIONNAIRE  DATA. 

After  every  hour  of  simulation,  controllers  were  asked  to 
complete  a  brief  questionnaire  which  contained  five  numerical 
scales.  These  scales  requested  estimates  of  workload,  self- 
assessed  performance,  busyness  during  the  simulation,  stress,  and 
the  degree  to  which  the  separation  standard  used  during  the  hour 
would  be  workable  in  the  home  facility.  This  last  scale  was 
written  in  such  a  way  that  a  high  numerical  score  meant  a  low 
level  of  workability.  The  closer  to  "l"  was  an  individual’s 
response,  the  more  workable  he  viewed  the  separation  standard. 

On  the  other  four  scales  higher  scores  meant  a  higher  perceived 
level  of  workload,  performance,  busyness,  and  stress 
respectively.  The  workload  scale  stood  alone  as  the  only  12- 
point  scale  while  all  the  rest  were  10  points.  This  was  because 
it  was  a  special  ATC  application  of  the  well  known  Cooper-Harper 
scale  long  used  in  aviation  for  the  evaluation  of  the  handling 
qualities  of  aircraft. 

Table  8  provides  an  overall  summary  of  the  response  means  on  the 
questionnaire . 


TABLE  8. 

QUESTIONNAIRE 

RESPONSE 

MEANS 

Scale 

- 

Separation 

Workload 

Performance 

Busyness 

Stress 

Workable 

1.5  nmi 

5.65 

(2.08) 

7.50 

(1.41) 

6.42 

(1.53) 

3.63 

(2.29) 

1.92 

(1.75) 

2.0  nmi 

6.04 

(2.53) 

7.00 

(1.71) 

6.45 

(1-91) 

3.66 

(2.69) 

1.75 

(1.09) 

Standard  Deviations  in  parentheses 


An  examination  of  table  8  leaves  one  with  a  picture  that,  on  the 
average,  controllers  felt  that  their  workload  was  moderate;  their 
performance  was  adequate;  they  were  busy  but  not  too  busy;  and 
both  separation  standards  were  workable  in  their  home  facility. 
There  also  does  not  seem  to  be  any  appreciable  difference  between 
the  two  separation  standards.  Each  of  the  means  in  table  8 
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represents  the  point  estimate  or  number  which  best  describes  all 
the  responses  made  to  a  question  within  each  separation 
condition.  It  is  a  very  broad  brush  and  may  have  missed 
information  of  interest.  However,  as  the  discussion  which 
follows  will  demonstrate  through  graphics  and  statistics,  the 
sweeping  overview  using  table  8  is  not  far  off. 

One  of  the  first  questions  that  is  often  asked  in  analyzing  data 
as  we  have  here  is  whether  or  not  there  are  any  trends  over  time 
which  may  be  a  function  of  learning  or  experience  in  the 
simulation.  Figure  17  presents  a  plot  of  the  mean  responses  to 
the  questionnaire  across  the  12  hours  of  simulation.  Just 
examining  the  figure  itself  can  be  useful.  Each  plotted  point 
represents  the  mean  responses  of  the  four  controllers.  An  effect 
resulting  from  the  differences  induced  by  separation  standard 
would  appear  as  a  sawtooth  pattern  across  the  graph.  For  the 
most  part,  there  does  not  seem  to  be  any  consistent  pattern  like 
this.  Workload,  performance,  and  busyness  appear  to  be 
consistently  moderate,  with  the  possible  exception  of  run  5  where 
things  went  wrong  and  controllers  ran  into  operational  problems. 
This  resulted  in  the  only  major  dip  in  their  personal  estimates 
of  performance.  The  reader  may  wish  to  go  back  to  the  observer 
data  and  compare  it  to  what  the  controllers  said.  Also,  a  great 
deal  of  information  can  be  obtained  from  the  correlation  matrix 
described  back  in  table  7.  For  the  most  part,  correlations 
against  run  sequence  number  indicate  little  or  no  trend.  This 
was  confirmed  with  a  multilinear  regression  using  both  the 
questionnaire  and  observer  data  as  independent  variables  and  the 
run  sequence  as  the  dependent  variable.  A  multiple  R=*. 48  was 
computed  which  would  indicate  a  moderate  relationship.  However, 
the  ANOVA  on  the  regression  was  not  significant  (F-1.27,  P>.05,df 
9,38).  Figure  17  shows  what  appears  to  be  trend  in  terms  of 
stress.  Stress  correlated  r»-.29  against  run  sequence.  It  looks 
as  though  everything  was  downhill  after  run  5.  Given  the  results 
of  the  regression  analysis,  this  is  not  a  significant  effect. 

For  all  intents  and  purposes,  there  were  no  significant  trends  in 
questionnaire  responses  over  the  course  of  the  experiment. 

What  was  also  very  interesting  were  the  relationships  of 
controller  and  observer  estimates  of  workload  and  performance. 
There  was  some  relationship  between  what  the  observers  saw  as 
workload  and  what  the  participants  perceived.  This  was 
demonstrated  by  a  mild  positive  correlation  of  r».36.  There  was 
no  relationship  whatsoever  between  observer  performance  ratings 
and  self  estimates  by  controllers  r— .04.  The  means  of  the 
controllers  estimates  were  very  consistent  and  may  have  reflected 
a  lack  of  objectivity  based  on  high  self  confidence,  which  is  a 
personality  trait  which,  of  necessity,  is  characteristic  of 
controllers.  The  observers  were  somewhat  more  objective  and 
their  ratings,  which  were  developed  on  a  run  by  run  basis, 
mediated  somewhat  by  the  positive  halo  that  observer  2  may  have 
felt  for  the  1.5  nmi  separation  standard. 
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FIGURE  17.  QUESTIONNAIRE  RESPONSES  RUN  SEQUENCE  1  TO  12 


A  multilinear  regression  analysis  was  computed  using  all  the 
questionnaire  data  and  observer  estimates  to  see  if  there  was  any 
significant  relationship  with  separation  standard  as  a  dependent 
variable.  This  did  not  seem  likely  since,  as  indicated  earlier, 
there  was  no  clear  sawtooth  pattern  in  figure  17  or  back  in 
figures  15  and  16,  the  observer  estimates.  The  regression 
generated  a  multiple  R=.35  which,  for  a  multiple  R,  is  not 
anything  to  be  greatly  impressed  with  unless,  of  course,  it  is 
significant  from  zero.  The  ANOVA  on  the  regression  indicated 
that  this  result  could  have  occurred  by  chance;  it  was  not 
anywhere  near  significance  with  an  F=.82  (P>.05,  df  7,40).  The 
controllers  who  participated  in  this  experiment  saw  no 
significant  difference  in  their  workload,  performance,  busyness, 
or  stress  levels  based  on  separation  standard  alone  (figure  18) . 

Looking  within  the  questionnaire  responses  themselves  and 
ignoring  the  separation  standards  under  which  they  were 
collected,  provides  some  insight  into  some  basic  concerns  about 
controller  behavior.  One  ongoing  problem,  which  is  well  known  in 
most  complex  command  and  control  systems,  is  stress.  Referring 
back  to  table  7,  it  is  interesting  to  see  what  correlates  with 
stress.  The  other  items  which  correlate  the  highest  were 
workload,  busyness,  and  workability.  The  latter  item, 
workability  correlates  positively  because  it  is  an  inverse  scale, 
the  higher  the  response  the  less  workable  the  situation  appeared. 
It  was  decided  to  see  whether  one  could  predict  stress  responses 
using  the  answers  to  other  scales.  Based  on  some  trial  and  error 
experimentation  with  the  data,  a  multiple  regression  was  finally 
computed  using  stress  as  the  predicted  variable  and  the  following 
independent  or  predictor  variables;  workload,  busyness,  and 
performance.  The  result  provided  a  multiple  R=.84  and  the  ANOVA 
on  the  regression  was  very  significant  (F=35.52,  Pc.001,  df 
3,44).  It  would  appear  that  the  mental  concept  of  stress  which 
the  controllers  held  during  the  experiment  included  elements  of 
workload,  busyness,  and  performance. 

POST-EXPERIMENT  INTERVIEW. 

The  four  controllers  and  the  two  observers  all  completed  an 
interview  at  the  conclusion  of  the  experiment.  This  was  the  last 
opportunity  to  explore  their  thinking  and  experiences  concerning 
the  impact  of  the  approach  separation  alternatives.  These  data 
were  qualitative  and  will  be  summarized  in  this  section.  A  copy 
of  the  interview  protocol  is  available  in  the  appendix. 

When  asked  if  they  could  use  the  reduced  separation  standard  in 
their  home  facility,  all  the  participants  and  observers  were  in 
complete  agreement.  They  indicated  that  they  could  use  the  1.5 
nmi  standard  and  that  safety  would  not  be  compromised.  When 
asked  if  it  would  influence  their  workload,  three  participants 
felt  that  the  1.5  nmi  standard  would  not  influence  their 
workload.  Only  one  indicated  that  it  would  reduce  how  hard  he 
had  to  work.  The  observers  were  also  divided.  One  could  foresee 
no  influence  while  the  other  said  the  reduced  standard  would 
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FIGURE  18.  RESPONSES  TO  THE  POST-RUN  QUESTIONNAIRE 


reduce  the  workload.  These  responses  are  consistent  with  the 
post-run  questionnaire  data  which  indicated  no  perceived  impact 
on  workload. 

All  the  participants  were  satisfied  with  the  familiarization 
training  they  received  concerning  the  simulation.  The  four 
responses  of  the  participants  on  the  scale  which  asked  them  to 
evaluate  simulation  realism  were  7,  7,  8,  and  9,  respectively, 
where  1  was  very  unrealistic  and  10  referred  to  very  realistic; 
the  two  observers  were  divided.  Observer  1  assigned  an  8  and 
indicated  that  it  was  a  very  good  simulation.  However, 
observer  1  stated  that  real  pilots  would  have  questioned  more, 
especially  if  given  clearances  that  seemed  out  of  tolerance  with 
the  norm.  Observer  2  assigned  a  realism  rating  of  4  with  the 
comment  that  no  simulation  can  be  that  realistic  without  live 
traffic. 

When  asked  what  they  found  most  difficult  during  the  12  hours  of 
simulation,  responses  were  varied.  Two  commented  that  the  lack 
of  a  designated  feeder  position  made  things  more  difficult  when 
activity  became  busy.  One  indicated  that  he  had  to  get  used  to 
the  way  the  aircraft  targets  turned,  which  was  somewhat  faster 
than  he  was  used  to.  The  last  participant  stated  that  learning 
to  work  with  controllers  he  had  not  worked  with  often  in  the  past 
was  the  most  difficult.  He  had  to  learn  what  he  could  expect 
from  them  so  that  he  could  plan  effectively.  One  observer 
pointed  out  pilot  errors  as  a  problem  which  increased  workload. 
When  asked  what  had  the  most  influence  on  how  hard  they  had  to 
work  in  order  to  maintain  performance,  three  out  of  four 
participants  cited  coworkers  as  the  key.  One  mentioned  his  ego 
and  how  controllers,  in  general,  like  a  challenge.  Only  one 
referred  to  the  aircraft  mix  of  airspeeds  and  types.  One 
observer  cited  pilot  performance?  the  other  echoed  participant 
concern  for  the  abilities  and  commitment  of  coworkers. 

When  asked  to  describe  his/her  approach  during  high  workload 
conditions,  participants  emphasized  the  importance  of  planning 
and  flexibility.  Strategies  included  switching  runways, 
adjusting  airspeeds,  and  the  use  of  vectors  to  keep  aircraft 
apart.  One  controller  noted  that  under  conditions  of  extreme 
workload,  he  shortens  his  radio  transmission  to  the  minimum 
necessary  for  adequate  communication.  All  the  participants 
stated  that  they  worked  their  traffic  the  way  they  normally  did 
and  used  the  same  techniques  that  they  used  every  day  at  their 
home  facility.  The  observers  commented  that  the  participants  did 
not  have  to  adjust  for  weather,  since  it  was  not  introduced  as 
part  of  the  problem.  They  also  did  not  have  to  focus  on  aircraft 
inside  the  outer  marker  as  much  as  they  would  when  working  actual 
traffic.  One  observer  commented  that  the  participants  were,  in 
essence,  following  experimenter  instruction  to  push  the  system  to 
its  limits  without  compromising  safety. 
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THE  RESULTS  AND  TRAFFIC  CAPACITY. 


After  questions  about  safety,  the  second  most  frequent  concern 
about  procedural  change  is  whether  or  not  it  influences  the 
system  in  any  positive  way.  With  today's  crowded  skies,  we  are 
constantly  looking  for  methods  to  increase  capacity  of  the  system 
without  compromising  safety.  In  a  previous  section,  it  was 
demonstrated  that  there  was  a  small  but  finite  increase  in  the 
number  of  landings  using  the  1.5  runi  standard.  A  mean  difference 
of  3.67  aircraft  per  hour  was  identified,  which  represented  a  5.7 
percent  shift  in  capacity.  One  might  want  to  establish  some 
assumptions  and  speculate  on  the  possible  impact  this  might  have 
on  a  given  high  volume  airport. 

The  assumptions  are  relatively  straight  forward  and  could 
potentially  lead  to  a  final  conclusion  which  was  either  too 
conservative  or  too  generous.  First,  one  must  assume  a 
relatively  consistent  demand  for  the  use  of  the  airspace  by  an 
aircraft  mix  represented  by  those  in  the  simulation.  Next,  we 
would  have  to  assume  relatively  consistent  weather  and  lack  of 
equipment  outages  or  malfunctions  which  might  tend  to  curtail 
operations.  Finally,  we  would  have  to  assume  that  controller 
skill,  motivation,  and  performance  were  well  represented  in  the 
simulation  sample  and  could  be  expected  in  a  field  setting.  . 
Controllers,  in  general,  are  highly  motivated  achievers,  and  this 
last  assumption  may  be  the  least  speculative  of  them  all. 

Given  these  assumptions,  the  potential  impact  on  capacity  for 
different  time  periods  can  be  estimated  by  adding  5.7  percent  to 
the  figures  that  would  occur,  assuming  a  base  landing  frequency 
of  64.33  aircraft  per  hour,  as  seen  in  the  simulation.  What  this 
would  look  like  is  seen  in  figure  19,  "Traffic  Capacity  Projected 
Landing  Rates."  This  figure  is  displayed  in  thousands,  so  the 
frequencies  can  be  determined  by,  moving  the  decimal  points  three 
places  to  the  right.  Over  a  year's  time,  changing  the  separation 
standard  could  mean  the  landing  of  595,680  aircraft  as  compared 
to  563,530  using  the  2  nmi  standard.  Figure  20  summarizes  the 
potential  gain  in  landings  over  the  time  periods  with  a  projected 
annual  increase  of  32,150  landings.  Admittedly,  this  is  very 
speculative  and  the  reader  should  evaluate  the  projections  based 
on  the  realities  of  the  airspace  system. 


CONCLUSIONS 


This  was  a  small  sample  study  based  on  observations  and  feedback 
from  four  operational  controllers  and  two  observers.  Results  are 
indicative  rather  than  conclusive.  Given  these  qualifications 
the  following  may  be  drawn  from  the  data. 

1.  Controllers  were  able  to  land  5.7  percent  more  aircraft  using 
the  1.5  nautical  mile  (nmi)  standard. 
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FIGURE  19.  TRAFFIC  CAPACITY  PROJECTED  LANDING  RATES 
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FIGURE  20.  PROJECTED  CAPACITY  GAINS  AIRCRAFT  LANDING  FREQUENCIES 


2 .  There  were  no  decreases  in  safety  using  the  1 . 5  nmi  standard 
as  demonstrated  by  violation  frequencies  and  the  aircraft 
proximity  index  (API) . 

3.  Once  controllers  became  experienced  in  the  simulation,  there 
were  no  differences  in  the  frequency  of  longitudinal  violations 
across  the  two  separation  standards. 

4.  There  were  no  reported  or  observed  differences  in  controller 
workload  or  performance.  Controllers  were  very  supportive  of  the 
possible  change  to  the  1.5  nmi  separation  minimum. 
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APPENDIX 


TRAINING/ FAMILIARIZATION  PLAN 


Primary  Objective. 

Familiarize  the  participant  controllers  with  the  Air  Traffic 
Control  Simulation  Facility  and  the  airspace  geometry  which  it 
simulates.  Ensure  that  participants  are  able  to  control  a 
moderate  level  of  traffic,  using  the  appropriate  procedures  and 
techniques . 

Enabling  Objectives. 

1.  Condition:  Given  a  routine  air  traffic  sample  of  ten  or  less 
aircraft  in  sector. 

a.  Task:  The  participant  maintains  communications  with 
aircraft  under  his/her  control  and  with  adjacent  controllers  as 
required  for  inter-sector  coordination. 

b.  Standard:  The  participant  employs  standard  radio 
telephone  procedure,  initiates  contact  to  obtain  required 
information  or  provide  information  and  directives,  and 
accomplishes  all  necessary  land  line  coordination  with  adjacent 
sectors . 

2.  Condition:  Given  a  briefing  and  documents  concerning 
operational  procedures . 

a.  Task:  The  participant  demonstrates  his/her  knowledge  and 
acceptance  of  these  procedures  through  verbal  discussion  with  the 
training  controller. 

b.  Standard:  The  training  controller  verifies  that  the 
participant  has  a  working  knowledge  of  procedures. 

3.  Condition:  Given  air  traffic  sample  of  ten  or  less  aircraft 
of  mixed  types  and  flightpaths  where  potential  conflicts  are 
preprogrammed . 

a.  Task:  The  participant  maintains  radar  surveillance, 
anticipates  and  identifies  potential  conflicts,  and  issues 
amended  clearances. 

b.  Standard:  During  a  1-hour  simulation  the  participant, 
controller  does  not  allow  more  than  two  violations  of  the 
horizontal  separation  standard  of  aircraft  within  the  vertical 
separation  envelope,  and  in  no  case  are  the  violations  allowed  to 
progress  to  a  point  closer  than  miles  of  separation. 

4.  Condition:  Given  an  air  traffic  sample  of  15  or  fewer 
aircraft  of  mixed  types  and  flightpaths  where  conflicts  of 
separation  may  or  may  not  occur. 
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a.  Task:  The  participant  exercises  traffic  management 
techniques  to  minimize  delays  and  maintain  a  positive  and 
expeditious  traffic  flow. 

b.  Standard:  The  controller  maintains  positive  command  of 
the  traffic  flow  and  introduces  path  changes  only  where  necessary 
to  maintain  safe  efficient  traffic  flow. 

5.  Condition:  Given  this  training/ familiarization  program 
involving  briefings ,  printed  material,  and  "hands-on1*  control  of 
simulated  aircraft. 

a.  Task:  The  participant  controller  is  able  to  control 
traffic. 


b.  Standard:  The  participant  is  willing  to  state  the  he/she 
is  adequately  familiar  with  the  simulation  so  that  the  simulation 
itself  does  not  inhibit  his/her  performance. 
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TRAINING/FAMILIARIZATION  FLOW 


co 
H 
CO  Z 
CO  Q  W 

b  W  S 
<  at  u 
o 

(3 


C3 

Z 

h-i  b 
b  O 
b 

i-t  co 
OS  z 
MOM 
M  W  CO 
CO  CO  OS  H 

Id  H  D  z 
os  >  a  w 
=)  O  w  X 
a  as  u  5 
w  b  o  a 
cj  as  o 
a  a  b  a 
as  z 
b  < 


TRAINING 


F AA  TECHNICAL  CENTER 
CONTROLLER  ENTRY  QUESTIONNAIRE 


The  purpose  of  this  quest i onnai r e  is  to  find  out  something 
about  your  background  and  current  feelings  about  this  project 
in  order  to  better  understand  your  performance  during  the 
course  of  the  study.  All  information  is  collected  under  your 
letter  code  and  no  attempt  will  be  made  to  link  your  name  to 
to  the  answers  you  provide.  Welcome  to  the  Technical  Center 
and  thank  you  for  your  par t i c i pat i on ! 


1 . Par t i c i pant  Code:  _  Todays  Date: 


2. Your  total  experience  as  a  Controller  _  _ 

years  months 


3. Your  experience  at  your  current  facility  _  _ 

years  months 


4. How  long  have  you 
worked  parallel 

approaches?  _  _ 

""years"  months 


Please  indicate  your  level  of  agreement  with  each  of 
the  following  statements  by  circling  the  most  appropriate 
number  between  1  Strongly  Disagree  and  10  Strongly  Agree. 


I  freely  volunteered  to  participate 

i  n 

this 

project . 

1  2 

3 

4  5  6  7 

8 

9 

10  - 

Strongly 

Strongly 

Di sagree 

Agree 

I  currently 

am 

in  good  health. 

1  2 

3 

4  5  6  7 

8 

9 

10 

Strongly 

Strongly 

Disagree 

Agree 

During  the  last 

several  months.  I  have 

seen 

exper i enc  x  ng 

a  relatively 

high  level  of  stress. 

1  2 

3 

4  5  6  7 

8 

9 

10 

Strong.1  y 

Strongly 

Di sagree 

Agree 

CONTROLLER  SIMULATION  QUESTIONNAIRE 


INSTRUCTIONS 


PLEASE  COMPLETE  THE  FOLLOWING  QUESTIONS  AS  SOON  AS  YOU  HAVE  BEEN 
RELIEVED  FROM  YOUR  RADAR  POSITION.  YOUR  RESPONSES  SHOULD  FOCUS  ON 
ONLY  THE  WORK  THAT  YOU  HAVE  JUST  COMPLETED  IN  THE  LAST  HOUR. 

ALL  CONTROLLERS  EXPERIENCE  A  WIDE  VARIETY  OF  ACTIVITY  AND 
RESULTANT  WORKLOAD  DURING  THEIR  CAREERS.  IT  DOES  NOT  DETRACT  FROM 
YOUR  PROFESSIONALISM  IF  FOR  A  GIVEN  PERIOD  YOU  REPORT  VERY  HIGH 
OR  VERY  LOW  WORKLOAD.  ON  ALL  THE  QUESTIONS  WHICH  FOLLOW  FEEL  FREE 
TO  USE  THE  ENTIRE  NUMERICAL  SCALE  FOR  EACH  ANSWER.  BE  AS  HONEST 
AND  AS  ACCURATE  AS  YOU  CAN.  YOUR  NAME  IS  NOT  RECORDED  ON  THIS  OR 
ANY  OTHER  FORM,  AND  NO  ATTEMPT  WILL  BE  MADE  TO  ASSOCIATE  YOUR 
RESPONSES  WITH  YOU  AS  AN  INDIVIDUAL.  DATA  COLLECTED  WILL  BE  FOR 
RESEARCH  PURPOSES  ONLY.  THANK  YOU  FOR  YOUR  PARTICIPATION. 


POST  RUN  CONTROLLER  QUESTIONNAIRE 


PARTICIPANT  CODE _  DATE 

PARTNER'S  CODE  _  TIME _ 

RUN  NUMBER  _  RUNWAY 


1.  CHOOSE  THE  ONE  NUMBER  BELOW  WHICH  BEST  DESCRIBES  HOW  HARD 
YOU  WERE  WORKING  DURING  THIS  PERIOD: 


DESCRIPTION  OF  WORKLOAD 
_ CATEGORY _ 


RATING 
(CIRCLE  ONE) 


VERY  LOW  WORKLOAD-  ALL  TASKS  WERE  1 

2 

WERE  ACCOMPLISHED  EASILY  &  QUICKLY  3 


MODERATE  WORKLOAD-  THE  CHANCES  FOR  4 

5 

ERROR  OR  OMISSION  WERE  LOW  6 


RELATIVELY  HIGH  WORKLOAD-  THE  CHANCES  7 
FOR  SOME  ERROR  OR  OMISSION  WERE  8 
RELATIVELY  HIGH  9 


VERY  HIGH  WORKLOAD  -  IT  WAS  10 
BARELY  POSSIBLE  TO  ACCOMPLISH  11 
ALL  TASKS  PROPERLY  12 


2. RATE  YOUR  PERFORMANCE  CONTROLLING  TRAFFIC  DURING  THE  PAST 
HOUR.  CIRCLE  THE  NUMBER  WHICH  BEST  DESCRIBES  HOW  WELL  YOU 
THINK  YOU  DID. 

123456789  10 

AVERAGE  EXCELLENT 


3.  WHAT  FRACTION  OF  THE  TIME  WERE  YOU  BUSY  DURING  THE  PERIOD 
YOU  WERE  CONTROLLING? 


12  3 

SELDOM  HAD 
MUCH  TO  DO 


5  6  7  8  9  10 

FULLY  OCCUPIED 
AT  ALL  TIMES 


4.  RATE  THE  DEGREE  TO  WHICH  YOU  FOUND  THIS  CONTROL  PERIOD 
STRESSFUL!  CIRCLE  THE  NUMBER  BELOW  WHICH  BEST  DESCRIBES 
HOW  YOU  FELT. 


123456789  10 

LOW  HIGH 

STRESS  A-6  STRESS 


5.  WOULD  THE  CONDITIONS  OF  THIS  PAST  HOUR(volume  of  traffic, 
procedures,  geography,  separation  minimum)  BE  WORKABLE  AT 
YOUR  FACILITY?  CIRCLE  THE  NUMBER  BELOW  WHICH  BEST 
DESCRIBES  THE  STRENGTH  OF  YOUR  AGREEMENT. 


1 

STRONGLY 

AGREE 


5  6 


9  10 

STRONGLY 

DISAGREE 


6.  DID  YOU  AND  YOUR  PARTNER  FOR  THIS  PAST  HOUR  ESTABLISH  ANY 
STRATEGY  OR  AGREEMENT  ABOUT  WHO  WOULD  DO  WHAT? 

IF  YOUR  ANSWER  IS  YES,  PLEASE  DESCRIBE  BRIEFLY  BELOW  WHAT 
YOU  DECIDED  TO  DO  EVEN  IF  THE  ARRANGEMENT  WAS  UNSPOKEN. 

BE  SPECIFIC  ABOUT  WHO (USE  THE  LETTER  CODES)  WAS  TO  DO 
WHAT. 


7.  IS  THERE  ANYTHING  ELSE  THAT  HAPPENED  THIS  PAST  HOUR  WHICH 
YOU  FEEL  MIGHT  HELP  US  UNDERSTAND  THE  RESULTS?  ANY 
COMMENTS  YOU  HAVE  AT  THIS  POINT  WOULD  BE  VERY  WELCOME. 
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INTERVIEW  PROTOCOL 


PARTICIPANT  CODE 


DATE 


1.  Could  you  actually  use  the  reduced  separation,  at  your 
facility,  to  land  more  aircraft?.  In  your  opinion,  are 
present  levels  of  safety  retained  when  the  reduced 
separation  is  used? 


2.  In  your  opinion,  is  controller  work  effort  increased, 
decreased,  or  unchanged  by  the  1.5NM  standard? 


4.  How  adequate  do  you  feel  was  the  training/f amiliarization  you 
received  before  we  started  collecting  data? 


5.  Could  you  assign  a  number  from  1  (very  poor)  to  10  (very 
good)  which  describes  how  adequate  the  training/ 
familiarization  was  for  you? 


A-8 


1 


What  did  you  find  was  the  most  difficult  for  you  to 

accomplish  during  the  last  _  runs?  (NOTE:  If  respondent 

has  difficulty  --  province  examples,  i.e.,  planning, 
navigation,  identifying  conflicts,  route  changes,  vectoring, 

coordination,  etc.,  - >  use  examples  only  if  necessary.) 

PROBE  FOR  EXPLANATION! 


Reflect  back  on  your  own  experience  both  as  an  active 
controller  and  in  this  simulation.  We  would  like  to  draw  on 
your  expertise!  What  do  you  believe  influences  how  hard  you 
have  to  work  in  order  to  maintain  your  performance? 


After  the  respondent  has  spoken  for  awhile  — 
probe  to  identify  if  he/she  has  a  verbal izable 
Internalized  performance  standard. 


Every  controller  establishes  strategies  or  common  ways  of 
dealing  with  traffic. 


WOULD  YOU  DESCRIBE  YOUR  APPROACH  DURING  HIGH  WORKLOAD? 


DID  YOU  CHANGE  YOUR  REGULAR  STRATEGIES  IN  ANY  WAY  IN  ORDER  OT 
CONTROL  DURING  THE  SIMULATIONS? 


Is  there  anything  else  you  think  we  should  know  that  has  not 
been  already  covered? 


OBSERVER  EVALUATION  FORM 


RUN  NO  _  DATE 

PARTICIPANTS 

OBSERVED  TIME 


1.  BELOW  PLEASE  CIRCLE  THE  NUMBER  WHICH  BEST  DESCRIBES  HOW 
HARD  THE  CONTROLLER  TEAM  WAS  WORKING  DURING  EACH  HALF 
HOUR  OF  THIS  RUN. 

FIRST  HALF 

1  VERY  EASY 

2 

3 

4 

5 

6 

7 

8 
9 

10  VERY  HARD 

2.  BELOW  PLEASE  CIRCLE  THE  NUMBER  WHICH  BEST  DESCRIBES  CONTROLLER 
TEAM  EFFECTIVENESS  DURING  EACH  HALF  HOUR  OF  THIS  RUN. 

FIRST  HALF 

1  AVERAGE 

2 

3 

4 

5 

6 

7 

8 
9 

10  EXCELLENT 


In  this  space  please  count  or  tally  the  number  of  times  either 
controller  exhibits  behaviors  which  are  not  related  directly  to 
control  duties (ie  talks  to  observers,  looks  away  from  the  display) 


First  Half 

Second  Half 

Place  any  comments  or  other  observations  on  the  back  of  form. 

SECOND  HALF 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


SECOND  HALF 

1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


Table  A-1 

PARALLEL  CONFLICT  DISTRIBUTIONS 


Run  1 

HORIZONTAL 

SEPARATION 


2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

0 

0 

0 

0 

0, 


0 

9 

8 

7 
6 

5 

4 
3 
2 
1 
0 
9 

8 
7 

6 

5 


Table 

Violations  and  Separations 
Parallel  Conflicts 


1 

3 
1 
1 
1 

4 


1 

3 

2 

1 

3 


1 

4 


3 

3 

3 


1 

2 

1 

3 

1 


3 

3 

1 


0  100  200 

TOTAL  VIOLATIONS 
57 


300  400  500  600 

VERTICAL  SEPARATION 


700  800 


VIOLATIONS  AND  SEPARATIONS 
RUN  2  PARALLEL  CONFLICTS 


HORIZONTAL 
SEPARATION 
2.0 
1 . 9 
1.8 
1.7 
1.6 
1.5 
1.4 
1.3 

1.2  1 

1.1 
1.0 
0.9 

0.8  1 

0.7 
0.6 
0.5 


1 

1 


1 

1 


5 

4 

1  1 

3 

1  1 


1 


0  100  200 


TOTAL  VIOLATIONS 
24 


300  400  500  600 

VERTICAL  SEPARATION 


700  800 


900 


900 
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PARALLEL  CONFLICT  DISTRIBUTIONS 


VIOLATIONS  AND  SEPARATIONS 
RUN  3  PARALLEL  CONFLICTS 


HORIZONTAL 

SEPARATION 

2.0 

1.9  1 

1.8  3 

1.7  5 

1.6  2 

1 . 5 
1.4 
1.3 

1.2  1 

1.1 
1.0 
0.9 
0.8 
0.7 
0.6 
0.5 


0  100  200 

TOTAL  VIOLATIONS 
66 


300  400  500  600 

VERTICAL  SEPARATION 


700  800 


VIOLATIONS  AND  SEPARATIONS 
RUN  4  PARALLEL  CONFLICTS 

HORIZONTAL 
SEPARATION 
2.0 
1.9 
1.8 
1.7 
1.6 
1.5 

1.4  1 

1.3 

1.2 

1.1  1 
1.0 
0.9 
0.8 
0.7 
0.6 
0 . 5 


3  2  1 

13  1 

1  2  1 
3 

1  1 

1 

1 


0  100  200 

TOTAL  VIOLATIONS 
2,4 


300  400  500  600 

VERTICAL  SEPARATION 
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700 


800 


900 


900 


PARALLEL  CONFLICT  DISTRIBUTIONS 


VIOLATIONS  AND  SEPARATIONS 
RUN  5  PARALLEL  CONFLICTS 


HORIZONTAL 

SEPARATION 

2.0 

1.9  3 

1.8  1 

1.7  3 

1.6  1 

1.5  2 

1.4 
1.3 
1.2 
1.1 
1.0 
0.9 
0.8 
0.7 
0.6 
0.5 


2 

1 


6 

1 

1 

5 

5 

3 

2 

1 


1 


1 


0  100  200 

TOTAL  VIOLATIONS 
60 


300  400  500  600 

VERTICAL  SEPARATION 


700 


800 


900 


VIOLATIONS  AND  SEPARATIONS 
RUN  6  PARALLEL  CONFLICTS 


HORIZONTAL 

SEPARATION 

2.0 

1.9 

1.8 

1.7 

1.6 

1.5 

11  4  1 

5  1 

2  3  2 

1  2 

1 


V  •  o 

0.7 

0.6 

0.5 


1.4 

1.3 

1.2 

1.1 

1.0 

0.9 


0  100  200 

TOTAL  VIOLATIONS 
27 


300  400  500  600 

VERTICAL  SEPARATION 
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700 


800 


900 


PARALLEL  CONFLICT  DISTRIBUTIONS 


VIOLATIONS  AND  SEPARATIONS 
RUN  7  PARALLEL  CONFLICTS 


HORIZONTAL 

SEPARATION 

2.0 

1.9  1 

1.8  4 

1.7  3 

1.6 
1.5 
1.4 
1.3 
1.2 
1.1 
1.0 
0.9 
0.8 
0.7 
0.6 
0.5 


0  100  200 

TOTAL  VIOLATIONS 
39 


300  400  500  600 

VERTICAL  SEPARATION 


700  800 


900 


VIOLATIONS  AND  SEPARATIONS 
RUN  8  PARALLEL  CONFLICTS 


HORIZONTAL 

SEPARATION 

2.0 

1.9 

1.8 

1.7 

1.6 

1.5 

1.4 

1.3 

1.2 

1.1  1 

1.0 

0.9  2 

0.8 
0.7 
0.6 
0.5 


2 


0  100  200 

TOTAL  VIOLATIONS 
29 


300  400  500  600 

VERTICAL  SEPARATION 
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700 


800 


900 


PARALLEL  CONFLICT  DISTRIBUTIONS 


VIOLATIONS  AND  SEPARATIONS 
RUN  9  PARALLEL  CONFLICTS 


HORIZONTAL 

SEPARATION 

2.0 

1.9  4 

1.8  3 

1.7  1 

1.6  3 

1.5  2 

1.4  2 

1 . 3 
1.2 
1.1 

1.0  1 

0.9 
0.8 
0.7 
0.6 
0.5 


6  2 
3  2 

6 
3 


0  100  200 

TOTAL  VIOLATIONS 
76 


300  400  500  600 

VERTICAL  SEPARATION 


700  800  900 


VIOLATIONS  AND  SEPARATIONS 
RUN  10  PARALLEL  CONFLICTS 


HORIZONTAL 

SEPARATION 

2.0 

1.9 

1.8 

1.7 

1.6 

1.5 

1.4 

1.3  1 

1.2 
1.1 
1.0 
0.9 
0.8 
0.7 
0.6 
0.5 


0  100  200 

TOTAL  VIOLATIONS 
21 


300  400  500  600 

VERTICAL  SEPARATION 
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700 


800 


900 


PARALLEL  CONFLICT  DISTRIBUTIONS 


VIOLATIONS  AND  SEPARATIONS 
RUN  11  PARALLEL  CONFLICTS 


HORIZONTAL 
SEPARATION 
2.0 
1.9 
1 . 8 

1.7  1 

1.6  1 

1.5  3 

1.4  2 

1.3  2 

1.2  1 

1.1 
1.0 
0.9 
0.8 
0.7 

0.6  1 

0.5 


1 


1 


1 

1 

1 


1 


2 

4 

4 


1 


1 

5 

4 

1 


5 

3 

1 

1 


3 

1 

1 


0  100  200 

TOTAL  VIOLATIONS 
69 


300  400  500  600 

VERTICAL  SEPARATION 


700 


800 


900 


VIOLATIONS  AND  SEPARATIONS 
RUN  12  PARALLEL  CONFLICTS 


HORIZONTAL 

SEPARATION 

2.0 

1.9 

1.8 

1.7 

1.6 

1.5 

1.4  1 

1.3  1 

1.2  1  2 

1.1 
1.0 

0.9  1 

0.8  1 
0.7 
0.6 
0 . 5 


2  1 

2  2  5 

1 

3 

3 


1 


0  100  200 

TOTAL  VIOLATIONS 
30 


300  400  500  600 

VERTICAL  SEPARATION 


700  800 


900 
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AN  AIRCRAFT  PROXIMITY  INDEX 


FURPOflB 

The  purpose  of  the  Aircraft  Proximity  Index  is  to  provide  a  measure 
of  the  seriousness  of  a  near  miss  between  two  aircraft.  It  is 
based  on  the  closest  vertical  and  horizontal  distances  during  a 
conflict,  and  it  provides  a  number  which  reflects  the  loss  of  safe 
separation. 


BACKGROUND 

A  conflict  is  defined  as  the  absence  of  safe  separation  between 
aircraft  flying  Instrument  Flight  Rules  (IFR) .  At  its  simplest, 
safe  separation  requires:  (a)  The  aircraft  must  be  laterally 
separated  by  3  NM  or  5  NM,  depending  on  distance  from  the  radar, 
OR  (fc)  vertical  separation  by  1,000  or  2,000  feet,  depending  on 
altitude  or  flight  level.  (There  are  refinements  of  the  above 
rules  that  take  into  consideration  the  fact  that  one  aircraft  may 
be  crossing  behind  another,  that  an  aircraft  has  begun  to  climb  or 
descend  from  a  previous  altitude  clearance,  and  special  Nwakes  and 
vortices'*  restrictions  for  aircraft  in  trail.  These  special  cases 
will  not  be  discussed  here.)  A  finer  distinction  than  the  simple 
presence  or  absence  of  a  conflict  is  often  needed,  some  conflicts 
are  more  serious  (potentially  dangerous)  than  others.  Possible 
applications  for  such  an  index  include  the  evaluation  of  Near 
Midair  Collision  Reports  (NMAC's),  analysis  of  the  "Conflict  Alert 
-  Immediate  Alert  Summary"  also  known  as  the  Quality  Assurance 
Program  or  "Snitch",  and  research  studies  where  impact  on  safety 
is  an  important  consideration. 

The  need  for  a  quantitative  evaluation  of  separation  violations  is 
most  strongly  felt  in  the  analysis  of  real  time  air  traffic  control 
simulations.  In  many  such  studies  it  is  necessary  to  deliberately 
introduce  errors  (conflicts)  to  establish  system  safety  and  to 
measure  the  capability  of  the  new  system  (controllers,  hardware, 
software,  and  procedures)  to  maintain  or  to  re-establish  safety. 

The  most  obvious  measure  of  proximity  in  the  three  dimensional 
world  of  air  traffic  is  the  slant  range  distance  between  two 
aircraft,  but  it  is  of  limited  value.  A  slant  range  of  1,000  feet 
is  considered  safe  if  it  is  1,000  vertically  and  zero  horizontally, 
while  the  same  1,000  foot  slant  range  is  unsafe  if  the  numbers  are 
reversed. 
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The  Aircraft  Proximity  Index  (API)  provides  a  weighted  value  for 
each  conflict  that  which  is  based  on  both  vertical  and  horizontal 
separation/  but  considers  their  contribution  to  safe  separation 
differently. 

COMPUTATIOM 

The  API  is  designed  to  range  from  100  for  a  mid-air  collision  to 
0  for  the  virtual  absence  of  a  technical  conflict.  A  linear 
decrease  in  distance  between  the  aircraft/  either  vertically  or 
horizontally/  increases  the  API  exponentially  by  the  power  of  2. 

The  basic  formula  for  API  is: 

API  -  (1/000  -  Dy)2  *  (3.00  -  D„)2  /90000 

Oy  *  vertical  distance  between  a/c  (in  feet) 

0N  *  horizontal  distance  between  a/c  (Naut.  Miles  (6,076')) 

API  -  0  when  Dy  »>  1,000  OR  when  D„  ->  3.0  NM 

Computations  are  done  to  round  off  the  API  to  the  nearest  integer: 

API  *int(  (1,000  -  Dy)2  *  (3.00  -  D,)2  /90000+  0.5) 

Tables  IA  and  IIA  give  examples  of  the  values  produced  by  the 
formula.  Figures  1A  and  2A  show  a  contour  and  three  dimensional 
plot,  respectively. 


APPLICATION 

In  recent  terminal  area  simulation  studies  at  the  Technical  Center, 
API  was  computed  whenever  a  conflict  occurred.  In  the  Atlanta  and 
Dallas  /  Fort  Worth  simulations  this  was  presumed  to  be  when  two 
aircraft  have  less  than  1,000  feet  of  vertical  separation  AMD  less 
than  3.0  miles  of  horizontal  separation  unless  both  were  on  the  ILS 
localizer.  API  was  computed  once  per  second  during  the  conflict, 
and  the  largest  value  computed  assigned  to  the  conflict. 

API  is  a  dynamic  value  over  time.  Under  most  conditions  the 
occurrence  of  conflict  should  see  its  value  start  small  and  build 
up  quickly  or  slowly  depending  speeds,  climb  or  descent  rates  and 
the  geometry  of  the  interaction.  In  the  parallel  runways 
situation,  the  application  of  radar  separation  standards  (and  the 
computation  of  API)  begins  when  one  a/c  deviates  from  the 
localizer.  With  runways  separated  by  3,000  feet,  an  initial 
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TABLE  IA 

API  VALUES  FOR  SELECTEO 
COMBINATIONS  OF  VERTICAL  ANO 
HORIZONTAL  SEPARATION 


TABLE  1.  TYPICAL  VALUES: 

VERTICAL 

DISTANCE  HORIZONTAL  DISTANCE  IN  NAUTICAL  MILES  (1  NM  »  6076') (CH) 
IN  FEET 


( Dv) 

3 

2.5 

2.0 

1.5 

1.0 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

.05 

.01 

1000 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

900 

0 

0 

0 

0 

0 

0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

800 

0 

0 

0 

1 

2 

2 

2 

2 

3 

3 

3 

3 

3 

4 

4 

4 

700 

0 

0 

1 

2 

4 

4 

5 

5 

6 

6 

7 

7 

8 

8 

9 

9 

600 

0 

0 

2 

4 

7 

8 

9 

9 

10 

11 

12 

13 

14 

15 

15 

16 

500 

0 

1 

3 

•  6 

11 

12 

13 

IS 

16 

17 

19 

20 

22 

23 

24 

25 

400 

0 

.A 

4 

9 

16 

18 

19 

21 

23 

25 

27 

29 

31 

34 

35 

36 

300 

0 

l 

5 

12 

22 

24 

26 

29 

31 

34 

37 

40 

43 

46 

47 

49 

200 

0 

2 

7 

16 

28 

31 

34 

38 

41 

44 

48 

52 

56 

60 

62 

64 

100 

0 

2 

9 

40 

36 

40 

44 

48 

52 

56 

61 

66 

71 

76 

78 

30 

-0- 

0 

3 

11 

25 

44 

49 

54 

59 

64 

69 

75 

81 

87 

93 

97 

99 

A-20 


TABLE  IIA 

EXTENDED  RANGE  VALUES  OF  API 


Dv 

API 

Oh 

ov 

API 

1>V 

apt 

3.0 

1000 

0 

1.0 

667 

5 

.05 

667 

L  1 

3 . 0 

0 

0 

1000 

0 

0 

1.0 

1.0 

500 

333 

11 

20 

.05 

.05 

500 
33  3 

24 

43 

2.0 

2.0 

2.0 

667 

500 

333 

1 

3 

5 

1.0 

1.0 

1.0 

250 

100 

0 

25 

36 

44 

.05 

.05 

.05 

250 

100 

0 

54 

73 

97 

2.0 

2.0 

250 

100 

6 

9 

.5 

.5 

667 

500 

8 

17 

.01 

.01 

667 

500 

11 

25 

2.0 

0 

11 

.5 

250 

39 

.01 

333 

44 

1  3 
1.5 
1.5 

667 

500 

333 

3 

6 

11 

.5 

.5 

.1 

100 

0 

667 

56 

69 

10 

.01 

.01 

.01 

250 

100 

0 

56 

30 

99 

1.5 

1.5 

1.5 

250 

100 

0 

14 

20 

25 

.1 
.1 
.  1 

500 

250 

100 

23 

53 

76 

0 

0 

0 

667 

500 

333 

11 

25 

44 

.1 

0 

93 

0 

250 

56 

0 

100 

81 

0 

0 

100 
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value  can  be  as  high  as  70 ,  whils  5,000  foot  separation  can  produce 
an  API  of  53.  Subsequent  changes  depend  on  relative  a/c 
trajectories  and  controller  intervention. 

API  is  not  intended  as  an  absolute  measure  of  system  safety.  That 
is,  there  is  no  prescription  to  say  that  systems  which  produce 
API's  of  50  or  less  are  "safe."  The  proper  use  of  API  is  in 
comparing  similar  systems,  and  that  configurations  that  produce 
significantly  higher  API's  are  more  likely  to  be  inherently  less 
safe  than  those  that  produce  lover  values. 

For  a  thorough  understanding  of  simulation  data,  it  is  necessary 
to  have  at  least  one  condition  which  reflects  present  operations 
and  can  be  considered  a  "safe  standard"  for  comparison  purposes. 
Thus  a  new  configuration  which  is  not  less  safe  than  the  "safe 
standard"  may  be  acceptable. 

DX8CU88X0M 

While  any  set  of  weights  is  somewhat  arbitrary,  even  arbitrary 
weights  can  be  more  useful  than  a  simple  count.  API  facilitates 
the  identification  of  the  more  serious  (potentially  dangerous) 
conflicts  in  a  data  base  where  many  conflicts  are  present. 

While  the  index  is  not  intended  as  a  measure  of  acceptance  risk, 
it  meets  the  need  to  look  at  aircraft  safety  in  a  more 
comprehensive  way  than  simply  counting  conflicts  or  counting  the 
number  of  aircraft  that  came  closer  than  200  feet,  or  some  other 
arbitrary  value. 

It  may  or  may  not  be  of  use  in  comparing  conflicts  in  dissimilar 
environments.  An  API  of  85  in  en  route  airspace  with  speeds  of 
600  kts  is  not  necessarily  equivalent  to  an  85  in  highly  structured 
terminal  airspace  with  speeds  under  250  kts.  The  computation  of 
API  for  some  of  the  larger  data  bases  of  operational  errors  may  be 
useful . 
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OBSERVER 

COMMENTS 
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01  R1  DC 


D/C  Establish  game  plan  -  where  to  start  stagger  -  who  will  be 
first  -  speed. 

D  Coming  back  on  speed  w  E381  -  good  stagger  going.  Bring  it 
back  to  160. 

C  watch  your  speed  on  D948. 

D  I've  got  702MB  doing  180  to  catch  up. 

C  Coming  across  W/D430 

D  Put  somebody  on  S  side  (E280)  not  enough  airplanes. 

D  D398  didn't  take  turn  I'll  go  N  to  get  in  behind  -  or 

speed,  sequence. 

C  How  about  AC33  on  s  side? 

D  You  got  D398  coming  back?  You  still  putting  AC33  down  here? 
M2941  jockey  speeds.  They're  speeding  up  -  I'll  pull  IMS  out 
and  put  him  right  back  in. 
c  Want  me  to  run  2  tight? 

D  You  got  809  in  there. 

C  D252 ' s  gonna  be  next  on  my  side. 

C  Keep  AC236  hi  -  put  him  on  my  side,  D  I'll  be  ok 

D  Need  a  more  room  on  H  -  C  you  can  pick  him  up. 

D  Id  AC197  @160  -  Come  back  to  150  for  a  minute, 

c  You're  gonna  cut  yourself  off  D  no,  I'm  going  N.  2  got  them 

too  tight  for  you.  Go  ahead  and  run  2  tight. 

D  D555 ,  gonna  be  next  since  I  received  that  one  up. 

The  leader  line  patch  reduced  distractions  and  provided  more  time 
to  control  AC  -  more  attention  to  separation.  Had  some  speed  and 
heading  problems,  but  better  than  previous  runs. 


Not  as  many  AC  on  S  side  at  beginning 

Missing  speed  reductions  on  s  side  (several  pull  out  and  back  in 
due  to  speed) 

AC33  MA? 

AC263  -  Heading? 


01-R1-DC 
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01  R2  CB 

Decide  on  game  plan  -  N  takes  lead,  use  180  till  it  get  busy 
C  Take  E543  in  first,  I'll  be  right  behind  you.  Did  you  clear 
him,  B  yes,  we're  going  back  to  the  LOC. 

B  Since  you  got  more  airplanes  you  come  on  in  and  I'll  hit  your 
gaps . 

B  Run  COA  tight  and  I'll  space  on  him. 

Once  stagger  was  established,  very  little  communication  between 
controllers  -  they  take  their  cues  from  the  other  final,  i.e.,  when 
one  is  turned/cleared  they  also  turn/clear. 

B  Run  337  close  on  yours  -  I've  got  a  H 
CB  Jockeying  speeds  to  maintain  interval 
C  Slow  one  coming  -  leave  extra  room. 

B  How  about  2  run  tight  on  myself  C,  you  can  ’t  -  oh,  yeah  if 

you’re  already  turning  run  2  tight  on  myself  now? 

D797  is  up  at  180K  if  you  want  to  increase  1891. 

Spacing  toward  end  of  run  increasing  (getting  tired?) 

S  side  not  running  as  tight  as  N  side  throughout  run  -  left 
comfortable  and  kept  separation  at  that  point. 

Appears  controllers  have  a  good  stagger  going  -  clear  to  standard 
separation  rather  than  pushing  for  reduced.  Very  relaxed.  Pilots 
getting  all  turns,  speeds,  altitudes.  Much  easier  on  controllers. 
More  AC  on  S  side  to  start. 

01-R2-CB 


01  R3  DB 

D  Establishes  first  and  second  AC 

B  Use  170? 

D  Wanna  put  somebody  down  here  -  put  D1241 
Come  on  in  -  just  keep  cranking  'em  in. 

we  probably  ought  to  come  back  to  160K  -  they’re  closing  a 
j-ittle  bit. 

B  You  might  speed  him  up  -  I’m  at  160 
I'll  pull  D1665  out,  we're  closing 
D  No,  just  reduce  him  to  150  -  it'll  work 

You  run  two  -  heck  you  don ' t  have  2  to  run 
B  I'll  put  AC  &  D  behind  that  Heavy. 

D  You  got  D540  coming  m  •  how  about  picking  up  speed 

Ineffective  coordination  on  B's  part  -  said  he  had  a  H  and 
would  leave  extra  room  -  D  did  not  acknowledge,  keep  stagger 
going  and  hao  to  jockey  for  separation  (laterally) 

B  6LV  -  kick  it  up  ten  -  D  complies 

D  How  about  running  2  on  D81H,  I'll  put  him  behind  M2959. 

B  Talking  about  speeds  who  to  increase/decrease 

Minimal  verbal  coordination  -  stagger  established  quickly.  ™  ~~ 
Controllers  relaxed  -  comment  "there  could  b**  more  traffic": 
casual  chit  chat  re-"pilots",  turns 

01-R3-DB 
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01  R4  AD 

Establish  initial  turn  on  point,  speed  of  160  to  keep  1.5 
Coordination  on  speeds  and  sequence  to  establish  stagger. 

A  You've  got  a  heavy  -  I'll  put  2  on  him 

A  You  put  2  on  my  heavy  D  is  he  going  next?  Watch  altitude  on 

643  he’s  not  coming  down. 

A  Watch  your  speed  on  D102 

D  You  putting  846  next? 

D  How  about  567  down  there,  I’ll  stay  high  behind  aao 
D  Asked  E764  if  she  had  been  cleared,  lined  up  on  9R  anyway, 
turned  back  to  90 

A  You  turning  in  W/D525.  D  -  he's  coming  still  got  his  speed 
to  close  gap.  I'll  pick  him  up  to  180  for  a  minute  -  I  was 
late,  we  got  altitude. 

A  I’ll  run  946  on  Metro  -  you  run  2  on  the  heavy.  D  I'm  having 
trouble  getting  2  over  there. 

D  wanna  put  two  on. . . . 

E132  -  Too  late  on  turn  -  back  out  ~  ~  ™ 

USA290  given  240  rolls  out  140  E32  brought  into  slot  USA290  back 
to  downwind. 

100%  effort  throughout  run 

Using  more  speed  control  increase/decrease  for  desired  separation 
Controllers  are  relaxed  -  not  having  to  worry  about  #  of  aircraft 
-  could  work  more  if  needed  -  May  see  (probable)  tighter  separation 
if  they  are  pushed. 

Left  extra  space  for  DC3  -  maximum  speed. 

Controller  D  comments  about  overtake  situation,  turns  and  claws 
aircraft  on  downwind. 

Controller  A  issues  more  headings  for  precise  turn  on 

More  coordination  on  speeds  to  run  tighter  separation 

D846  takes  wrong  turn  -  A/D  work  out  where  to  go 

More  interactions  between  A  &  D  to  run  2  behind  heavy  rather  than 

1  on  1 . 

01-R4-AD 
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01  R5  AB 

Establish  initial  turn  on  point  "out  a  little  bit" 

B  I'll  go  on  in  -  speed  160  or  1707 

A  I'm  gonna  run  2  on  E71(H)  I'll  run  2  on  E97(H)  also 

increased  M2957  10K  til  MKR 

A  You  want  to  put  one  of  those  down  here  -  B  no,  we're  just 
about  even 

A  I'm  late  on  that  one.  Stop  D30@40  til  we  get  back. 

B  D868  clearance  pull  out 

A  I  missed  it,  go  on  in  W/E650  M3320  paralled  to  final,  put  back 
on  downwind.  12  mile  gap  on  S  side,  running  tight  on  N  side. 
B  Run  2  on  my  heavy  -  get  back  in  the  game. 

A  Need  a  shade  more  room.  We're  gonna  be  side  by  side  B,  I'll 

pull  out,  you've  more  clearance  on  COA1033.  B  didn't  hear 
pulled  his  out 
A  Said  put  him  back. 

B  E675'  pull  out 

Stagger  finally  re-established  toward  end  of  run.  when  the  "can 
turned  to  worms"  flow  would  have  curtailed  arrivals  so  final  could 
catch  up. 

Stagger  was  lost  -  difficult  to  re-establish.  Feeder  position 
would  be  chastised  if  this  was  actual. 

Coordination  increasing  as  traffic,  pilot  errors  increase. 
Building  traffic  forces  controllers  to  work  feeder  as  well  as 
final,  distracts  from  focus  on  stagger  -  getting  behind  on  power 
curve 

Pilot  errors  increase  load,  gaps  are  missed,  ragged  final.  A 
behind  power  curve. 

ASE144  0281  D1077  AAL483 

E544  Due  to  traffic  load  on  bane.  D433  (traffic  on  lane) 

Pilots  taking  wrong  call  signs,  wrong  turns 
18337  on  lane  ROA1280  -  won't  capture 
D568  pulled  out  in  error 

AC286  took  wrong  heading  COA1290  wrong  heading 
Three  calls  to  M3280  to  increase  speed  10k  to  MKR 
Relaxed  team,  little  chit  chat,  verbal  coordination 
first  turn  on  8  outside  MKR,  smooth  entry  to  stagger 

01-R5-AB 
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01  Rb  BD 

D  Drive  em  to  10  before  we  start,  I'll  put  E140  first  -  game 
plan  established.  160K  ok,  let's  tighten  em  up. 

D  What  speed  you  using  B  170  D  that's  not  gonna  work 

D  E498  on  south  side  B  ok  I'll  jockey  ASE  a  little  bit 

D  Come  on  in  w/354  I'm  gonna  put  330  in  there  since  I  missed 

the  turn 

Watch  your  speed  on-  3  54 

Stagger  on  ASE163  since  yours  is  a  heavy 
D  Watching  and  prompting  B  when  aircraft  seem  to  take  wrong 
turns . 

B  Having  to  work  feeder,  taking  aircraft  to  pattern  distracts 
from  primary  duty  of  turning  on  final. 

B  Prompts  D  to  turn  aircraft  into  gap  when  another  one  missed 
turn. 

D  No  -  speeding  up  succeeding  aircraft.  Take  1061  27  I'm 

coming  in  with  EJA451. 

B  Behind  power  curve  because  he  is  working  feeder.  Dropped  3 
aircraft  -  back  in  the  game. 

Not  much  verbal  coordination  as  long  as  everything  is  running 
smooth 

Good  stagger  going,  speed  on  final  consistent  @160. 

DLH439/DAL291  take  wrong  headings  on  N  side 
ASE3241  -  1  on  line  -  controller  though  ACID  ASE324 
N3229  pull  OUt  EME3277  N706DS  EME3243 
D633  mo  join  D718  takes  wrong  turn 
Jockeying  speeds  to  close  gaps . 

Relaxed,  easy  chit  chat  -  poking  fun  at  speaking  errors. 

01-R6-BD 
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01  R7  DA 

who's  going  first  -  Plan  established.  Stagger  established  easily. 
D  Let's  use  170.  Put  somebody  down  here,  I  can't  get  there. 
Coordinating  speeds  to  maintain  interval.  You  running  the  heavy 
next? 

How  about  I  run  2 . 

Coordinate  next  turn  due  to  heavies  both  sides. 

Little  coordination  needed,  stagger  good,  traffic  moderately  heavy. 
North  side  getting  busy  -  coordination  to  put  one  on  9R. 

Traffic  picking  up  both  sides.  Both  controllers  vectoring  to 
pattern  but  it's  not  overwhelming,  not  detracting  from  final. 
Smooth  run.  Both  at  ease. 

Program  stopped  9  56  minutes. 


Good  flow  -  manageable  traffic  -  not  pushed  but  enough  to  keep 
final  pull 

Pilots  flying  well  -  no  misses  (EA280  as  I  write)  on 
heading/ altitude/speed 

ASE289  cleared  9R  joined  9L  -  got  to  9R  outside  MKR. 

0 1-R7-DA 


01  R8  DC 

C  Establishes  he  is  #1  writes  all  signs  on  pad,  altitude 
clearance 

D  No  use  of  scratch  pad.  C  tells  D  to  run  two  on  his  heavy 
(D22)  . 

"It's  gonna  be  a  little  wide  til  we  get  our  stroke  down.  Speed 

180  established.  C  has  6  aircraft  on  scope  -  no  longer  using 

scratch  pad.  AC385  MA  -  put  back  in  pattern.  C  tells  D  to  run 
2  on  E7931.  AC 5  turns  off  localizer  on  his  own.  D  -  I'm  coming 
back  to  160K .  Asks  C  if  he  can  get  there  w/E680  D  coordinates 
speed  and  plan  w/E680  D  coordinates  speed  and  plan  W/AC3305  - 
Juggles  speeds  to  regain  stagger.  D  notes  that  ground  speed  ?  is 
10k  faster  asks  if  he  is  using  160  -  C  yes  -  D  says  he  will 

increase  to  170  C  says  no,  he  is  at  170  elbows  to  160.  c 

sporadically  writing  call  signs,  alt. 

C  Asks  if  C  1280  going  in  -  D  says  yes  -  oh,  you've  got  a  heavy, 
go  on  in.  How  about  I  run  3  on  D868.  D  pushing  -  bare  3 
miles  (looks  more  like  2.5  in  trail  on  his  localizer/asks  C 
if  he  needs  more  room. 

C  Using  scratch  pad  w/altitudes.  1806  lites  come  on  -  right 
back  off. 

D  Running  tight  -  separation  appears  less  than  1.5  on  several. 

Question  again  on  speed  C  back  up  to  170. 

Talking  about  running  2  on  N  side  -  finally  decide  to  run  1  on  1 
EME2979  drifts  to  N  side  inside  MKR  -  MA  to  south  (George  says  it's 
a  software  gli*ch  that  won't  affect  results) 

ASE313  speed  increasing  D  had  been  reducing  to  150  -  pulled  out. 
Late  on  other  turns  because  of  distraction. 

01-R8-DC 
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01  R9  AC 

c  Using  scratch  pad  A  not  -  no  game  plan  established. 

A  who's  first  -  C  you  ASE383  10E  ATL  N  -  bound  -  no  call. 

No  verbal  coordination  -  A  running  4-5  mile  gaps  -  c  hits  then. 

A  Tells  C  ASE2054  fitting  gap.  C  uses  2  mile  range  mark/A  50 
mile 

C  Tightens  up  forcing  A  to  tighten.  A  asks  what  speed  A831  - 
use  160  from  now  on  Yes  A  says  run  ASE163  tight  on  yourself 
I  can't  get  in  there  with  the  heavy.  1910  a  starting  to  tell 
C  Where  to  put  aircraft.  N  side  closing  on  S  side  -  C  using 
170  again  A  -  have  you  got  a  plan  on  D848  (cutting  out  s 
side ) 

C  Yeah,  I'm  gonna  turn  back  right.  A  points  -  2  miles 

( separation) 

A  running  5  mile  gaps.  D848  drifts  N  off  localizer  C  asks  if 
established.  A  -  late  on  turns  running  5-6  mile  gaps,  using  speed 
to  catch  up.  Putting  base  further  out.  Attempting  to  tighten  up. 
C  has  quit  trying  to  force  A  into  running  tighten.  Both  sides  out 
of  TCA.  Running  a  mere  consistent  4  miles  each  side.  A  asks  again 
if  c  is  using  160  or  170. 

01-R9-AC 

.  01  RIO  AB 

A  Establishes  game  plan  before  traffic  starts.  Neither 

controller  using  scratch  pad.  A  tells  B  to  go  on  in  - 
establish  sequence  for  stagger. 

A  Using  5  mile  range  mark;  B  -  2  miles  Using  18JK.  A  tells 
B  if  he  can  get  in,  put  2  on  D160H.  A  says  he  will  leave  a 
gap  behind  ASE232  get  in,  put  2  on  D160H.  A  says  he  will 
leave  a  gap  behind  ASE232. 

A  coordinates  speed  adjustments.  A  running  consistent  5  mile 
gaps  -  B  filing  them- 

Second  (middle)  "half  of  run  showing  a  tighten  stagger,  closer  to 
4  miles. 

A  Discusses  speed  of  B's  DC3  -  will  give  a  shade  more  room. 

very  little  verbal  coordination  -  cures  taken  visually  and  from 
listening  to  other  side  issues  turns  -  this  is  normal. 

CTY  70A  MA  -  back  to  pattern.  Verbal  coordination  between  2 
continues  as  to  who  is  next.  A  praises  B  for  recovery. 

B  asks  if  A  if  he  is  going  to  put  D946  in  there  -  obvious  he  has 
missed  a  turn.  A  increases  speed  on  2  aircraft  to  reduce 
separation.  Both  controllers  relaxed  but  attentive,  feet  on  the 
floor,  leaning  on  console.  A  is  vectoring  to  pattern  rather  than 
pushing  separation.  Traffic  building  both  sides. 

Speed  rate  and  pitch  remains  constant. 

01-R10-AB 
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Both  using  5  m  range  marks.  D  reminds  using  2  mile  separation. 

D  -  I'll  put  E246  first.  A  I’ll  put  2  on  E71H.  D/A  staart  talking 
about  speeds  -  when  to  reduce.  Neither  using  scratch  pad. 

D  asks  A  who  is  putting  next.  Using  speed  of  180.  A  sits  w/feet 
on  floor  resting  elbows  on  console  D  "laid  back".  A  says  he'll  get 
head  out  of  butt  and  start  hitting  gaps.  Lots  of  verbal  exchange, 
not  all  related  to  run.  D  issues  turn  and  looks  away  from  scope 
while  he  finishes  clearance.  Has  arms  folded  across  chest, 
swinging  in  chair.  Appears  very  bored.  A  states  he  is  picking  up 
speed  since  he  is  consistently  behind  gap.  D  states  several  times 
"not  enough  traffic.  Lots  of  verbal  exchange  between  D/A. 

D  tells  A  EME3220  going  through  localizer.  Traffic  building. 

A  vectoring  to  pattern.  D  widen  downwind  to  accommodate  base  leg 
rather  than  going  to  pattern.  Later  half  of  second  half  D  quiets 
down,  A  seems  to  find  his  stroke.  D  is  sitting  up,  leaning  toward 
scope.  A  still  vectoring  to  pattern  but  turning  on  @16  -miles. 
Once  you  go  to  pattern  increases  work  load. 

01 -Rll -AD 
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D  Asks  if  B  putting  EME3203  first.  OK,  I'll  go  behind  you. 

D  -  5  mile  range  mark  B  -  2  miles.  Neither  using  scratch  pad. 

B  Missed  turn  on  E140  D  turns  back  out  and  makes  it  work. 

Lost  separation  E140/ASE133  D  makes  no  altitude  correction 
( doesn' t ) 

B  -  put  2  on  that  heavy.  D  -  come  on  in  there,  I'll  be  there. 

D  asks  if  B  wants  to  put  someone  of  S  side  -  No  D  seems  to  be 
paying  closer  attention  than  last  run.  D/B  running  tighter 
stagger.  More  aircraft  on  N  side.  D  says  he  needs  more  airplanes, 
having  a  hard  time  getting  them  there.  B  runs  his,  D  hits  gaps. 
D  not  always  giving  standard  separation  behind  heavy.  stagger 
looks  good  but  not  always  legal.  D  corrects  FLX777  did  not  join, 
pulled  out.  D  now  has  more  aircraft  than  B  and  giving  heavy  jet 
separation.  D  goes  to  pattern  W/760DS  has  one  outside  of  TCA . 

B  says  put  EME3277  on  N  side  (will  put  D  back  into  TCA) . 

D  back  in  TCA/B  out.  B  vectoring  to  pattern.  D632  speed  is  up 
overtakes  AA02009  inside  MKR.  D  out  of  pattern.  ASE300  makes  two 
wrong  turns,  turned  eastbound. 


01-R12-DB 
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Controller  B  turning  in  on  localizer  less  than  the  3  miles  on 
traffic  on  north  side  without  vertical  separation. 

Controller  much  more  relaxed. 

Traffic  volume  requires  the  final  controller  to  vector  base  leg 
traffic  to  his  downwind.  This  is  normally  done  by  the  feeder 
controller.  Observed  aircraft  on  north  wind  at  same  altitude  less 
than  standard  separation  in  trail. 


Controller  continue  to  adjust  to  simulation.  North  final  still 
allowing  loss  of  in  trail  separation  on  north  downwind.  Controller 
(North  Final)  still  having  to  vector  base  leg  traffic  to  downwind, 
stagger  holding  pretty  good. 

02  R1  BA 


02  R2  DA 

The  2  first  aircraft  turned  on  too  close.  The  final's  are  trying 
to  rush  unnecessarily.  The  stagger  and  separation  once  on  final 
is  good.  Problem  still  while  turning  on  localizer  inside  initial 
approach  fix's.  (Less  of  vertical  prior  to  established  on  loc . ) 

These  two  controllers  are  working  very  well  together.  Both  appear 
to  be  relaxing.  South  final  descended  an  aircraft  to  4500  instead 
of  3500  and  turned  in  for  a  stagger  with  the  north  aircraft  at 
5000'  less  than  3  miles 


I  don't  see  any  difference  in  technique  or  turn  on  separation  while 
using  1.5  vs . 2 

Controller  chemistry  remains  very  good  this  half. 

02  R2  DA 


02  R3  CA 

Controller  are  turning  the  first  3  or  4  aircraft  on  with  less  then 
than  3  miles.  After  they  are  established  they  have  the  legal 
stagger.  I  believe  this  is  the  result  of  them  trying  very  hard. 
Team  work  is  good.  Both  controllers  are  talking  with  each  other 
more.  The  2  mile  stagger  is  much  more  consistent  today. 


Controller  "C"  continues  to  let  his  base  leg  push  him  out  of  the 
TCA .  Controller  "A"  takes  some  of  his  base  leg  to  downwind 
allowing  him  to  stay  in  TCA.  Most  of  attention  continues  to  be  at 
the  turn  on  point.  A  couple  of  aircraft  could  have  had  their  speed 
adjusted  on  final  to  keep  2  miles.  For  the  majority  of  time  I 
believe  2  mile  or  greater  existed. 

02  R3  CA 
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Initial  turn  on  is  better. 

Controller  "C"  advised  "B"  to  run  2  allowing  "B"  to  catch  up. 

Interaction  good.  Both  controller  constantly  talking  to  one 
another . 


Controller  inadvertently  read  EAL  763  runway  9R  instead  of  9L  on 
initial  contact. 

c  -  read  EAL  327  runway  9R  instead  of  9L. 

Both  controller  using  speed  control  more  while  on  final.  Both 
controllers  attention  to  aircraft  already  on  final  is  much  better 
than  earlier. 

DAL339/NW824  too  close  on  stagger,  no  action  taken. 

Controller  C  constantly  turning  on  smug  (too  tight)  to  south  final 
traffic 


02  R4  BC 


02  R5  CA 

Interaction  between  controller  C-D  very  good.  Traffic  load  on  this 
problem  ideal  for  stagger  at  this  point.  (15  min  into  problem) 


DAL432/EME2978  -  -  N18336  was  dropped  and  controller  “C"  tried  to 
put  EME2978  in  his  space  and  was  late  and  would  have  worked  if 
controller  "C"  had  descended  EME2978  to  2700.  Traffic  overload 
caused  extra  attention  to  be  diverted  from  turn  on's  to  working 
pattern.  A  more  consistent  interval  was  established  when  traffic 
was  heavy. 

02  R5  CA 
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EME2979  was  given  turn  right  to  070  and  aircraft  went  to  270. 
Overall  this  half  controller  are  working  at  a  comfortable  level 
with  a  very  good  stagger  interval. 


Controller  "C"  recognized  he  could  not  hit  the  hole  originally  for 
EME2979  and  pulled  him  out.  35  minutes  into  the  problem  both 
controllers  are  running  a  good  interval  and  are  aware  of  the  final 
after  approach  clearance.  DAL403  missed  his  turn  and  disrupted 
controller  "As"  thought  pattern.  It  took  a  couple  aircraft  before 
he  regained  his  stroke.  EME3184  and  ASE274  were  turned  on  beside 
traffic  on  south  side.  A3E274  was  pulled  out  before  loss  of 
separation.  EME3184  lost  vertical  separation  with  AA02008  prior 
to  having  1  1/2  vector. 

02  R6  CA 
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"C"  took  initiative  and  called  initial  sequence  and  recommended 
speeds  to  accommodate  turn  on's. 

"C"  recognized  "B"  had  more  aircraft  and  advised  him  to  run  2. 

"B”  advised  "C"  that  he  would  run  a  4  1/2  mile  interval  and  for 
him  to  hit  each  hole. 

"C"  advised  B  of  speed  difference  on  final  and  recommended 
corrective  action. 

25  minutes  into  problem  "C"  is  still  fine  turning  final  with  10  kt 
speed  adjustments. 

C  requested  B  to  adjust  his  speed  to  save  stagger  and  heavy  jet 
separation  on  "Cs"  runway. 

No  less  than  3  pair  of  aircraft  would  have  been  legal  with  1.5  vs 
2.0  (They  were  a  little  less  than  2.0. 

02  R7  CB 
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1.  '‘A  took  the  initiative  and  stated  game  plan. 

2.  "A"  pointed  out  on  aircraft  that  was  high  on  his  side 
tO"B" . 

3.  "B"  advises  "A"  on  aircraft  on  his  downwind's 
ground  speed  in  varying. 

5:52  4.  "A"  advised  "3"  to  run  two  to  catch  up. 

5:55  5.  "A”  adjusts  one  aircraft's  speed  on  final  to  hold 

interval . 

5:55  6.  "A"  advised  "B"  to  run  2  on  his  heavy. 

5:56  7.  "B"  &  "A"  laugh  about  similar  call  signs  and  problems 
it  creates. 

5:58  -  "A"  advises  "B"  he  has  slowed  one  to  allow  additional 

spacing  on  "B'"s  traffic. 

6:00  -  "B"  missed  gap  earlier  and  filled  it  with  another  from 

the  downwind  (N321DW) .  If  "B"  had  descended  N321DW  to 
2700  he  probably  could  have  made  it  legal.  As  a  result 
the  overtaking  aircraft  on  the  north  side  lost  vertical 
separation  prior  to  horizontal. 

6:18  -  "A"  &  "B"  discuss  sequence  involving  a  heavy 

jet  on  each  side  reference  in  trail  separation  of 
successive  aircraft. 

02-R8-BA 


A-35 


6  :  53 

6:57 

NOTE 


7  :  15 


7:25 

7:28 

7:32 

7:33 

7:35 

the 
7  : 43 


02  R9  DB 

EAL139  St  EAL295  took  up  incorrect  headings, 
causing  both  to  be  pulled  out,  I  think  pilot  reversed 
call  Sign. 

"D"  advised  "B"  lets  go  back  to  170KT  final. 

I  have  noted  that  in  the  last  few  problems  that 
controllers  aren't  switching  runways  on  their  aircraft 
to  balance  load.  In  real  time  which  ever  side  had  the 
most  aircraft  would  put  one  or  two  on  the  other  runway 
in  a  stagger  on  himself.  I  believe  the  reason  to 
be  the  amount  of  confusion  by  the  simulator 
pilots.  (Note:  not  complaining  about  pilots). 

As  traffic  builds  "D11  advises  "B"  we  need  to 
tighten  them  up  a  little. 

Most  of  the  time  the  controller  have  been  using  a  180KT 
final.  This  works  if  the  tower  can  see  them  at 
the  O.M.  If  the  tower  can't  then  a  slower  final  would 
be  required  to  hold  an  appropriate  interval  inside  the 
O.M.  The  controllers  are  assuming  to  tower  see's  the 
traffic  at  the  O.M.  and  is  providing  visual  separation. 

Controller  "B"  is  running  a  little  wider  interval  then 
"  D  " ,  causing  excessive  separation  or  final.  "B"  does 
the  same  thing  in  ATL. 

"D"  asks  "A”  if  he  wants  to  put  DAC171  on  his  side  and 
was  advised  no  necause  of  the  confusion  with  the 
pilots . 

controller  "B"  has  considerable  more  aircraft  then  ”D" 
and  is  working  lots  harder  then  "D". 

Stagger  looks  good. 

”B"  continues  to  have  a  longer  downwind  (30  west),  ”D" 
asks  "B"  to  put  some  of  his  aircraft  on  his  runway  but 
"B"  does  not.  No  reason  given. 

Controller  "B"  went  back  to  using  180KT  to  one  and  has 
caused  the  stagger  to  go  less  the  2  miles.  He  has  not 
advised  "D”  who  thinks  he  is  using  170KT  final. 

Controller  D  &  B  appear  to  be  working  well  on  the  2.0 
stagger  at  the  turn  on  and  ignoring  it  after 
aircraft  are  established. 

The  final  is  back  in  the  TCA  and  both  controllers  are 
now  fine  tuning  the  interval  after  they  are  established. 
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ifiCkts. 


02  RIO  CD 

"D"  St  "C"  confer  to  reduce  final  speeds  to 
Both  talk  of  how  light  the  traffic  is. 

08:38  "D"  asks  "C"  which  aircraft  was  next.  The  base  leg 

traffic  or  one  downwind. 

08:42  "D"  &  C"  discuss  how  much  better  the  simulator  pilots 

after  getting  used  to  each  other. 

08:46  DALI 01  St  ASE200  were  turned  on  to  localizer  b  y 

"D"For  some  reason  DAL101  crossed  localizer  after  he 
was  established.  ASE200  was  given  an  intercept  heading 
and  paralled  the  final  north  of  course.  After  ASE200 
was  established  he  went  to  right  of  course  at  about  same 
place  as  DAC101  DID. 

08:58  DAL 10 09  on  south  localizer  swung  over  to  the  north  final 

about  10  out.  "C"  noticed  it,  but  took  n  o 

corrective  action  thinking  it  was  a  computer 

problem  or  pilot  problem. 

09:00  "C"  decided  he  would  pull  DAL1009  out  and  work 

him  again. 

« 

09:06  "C"  gets  behind  on  hitting  a  couple  holes  and  stagger 

separation  is  questionable. 

09:10  "C"  is  struggling  to  run  constant  interval.  Traffic  is 

relatively  light. 


02-R10-CD 
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09:48  "  B"  returns  ASE384  on  above  the  glideslope  @350.  He 

had  ample  opportunity  to  get  him  to  2700. 

09:52  "B"  is  behind  the  power  curve  running  too  large  of  gaps 

for  "C"  to  hit.  Traffic  is  light. 

10:05  "C"  advised  "B"  he  would  run  his  2  heavy  jets  back  to 

back  vs  heavy  -  non  heavy  -  heavy. 

10:07  "B”  asked  "C"  if  he  could  run  2  ASE ’ s  tight  and  "C"  said 

no  he  would  have  one  for  the  slot.  "B"  had  more 

traffic  then  "C" . 


10:10  "C"  noticed  one  of  "B"'s  aircraft  needed  to  slow  and 

advised.  "B"  slowed  traffic. 

10:15  Stagger  looks  good.  Consistent  intervals.  ' 


10:16 

advise 

"B" 

to 

run  two  on  his  heavy. 

10:18 

"C" 

tells 

"  B  " 

to 

run  two  to  help  "B"  catch 

up. 

10:22 

"C" 

tells 

"B" 

to 

speed  up  COA1082  and  "B” 

says  no  you 

need  to  slow  WAL85  down.  I  believe  "B”  is  correct. 

10:28  "B"  recognized  EME2978  stagger  was  not  going  to  work  and 

pulled  him  out. 

10:30  "B '*  gave  EME2978  heading  150  for  pull  out  and  aircraft 
flew  050. 

10:36  "C"  coordinated  with  "B"  to  run  two  BAW7231  and  "B" 

advised  he  cant't,  his  next  one  is  a  heavy  also. 

Concentration  is  being  made  at  turn  on  point  allowing 
stagger  to  close  too  much  just  outside  the  O.M. 
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X 


02  R12  AC 

11:20  Good  Working  relationships  between  A-c.  Both  are  talking 
to  each  other  and  it  shows  with  a  good  operation. 

11:22  "A"  is  descending  below  3500'  when  required  to  maintain 

vertical  separation  until  stagger  is  legal. 

11:25  "C"  request  EAL329  go  on  south  side  and  stagger  on  his 

own  traffic. 

11:35  "A"  is  descending  all  his  downwind  traffic  to  3000  for 

unknown  reason.  The  base  of  the  TCA  is  3500. 

11:45  "C"  advised  "A"  he  needed  to  slow  an  aircraft  he  (C)  had 

put  on  his  (A)  runway. 

11:46  "C"  advised  DAL539,  who  was  on  downwind  to  slow  to  180kts 

and  the  aircraft  turned  to  180. 

12:00  "C"  was  late  on  one  turn  on  and  it  affected  his  next 

three  turn  on's.  (made  him  late) 
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